Drizzle Automation

Merge lp://qastaging/~kalebral-deactivatedaccount/drizzle-automation/fix-with-show-status-flag into lp://qastaging/drizzle-automation

  1. fix-with-show-status-flag
  2. Merge into trunk
Proposed by Lee Bieber
Status: Merged
Merged at revision: not available
Proposed branch: lp://qastaging/~kalebral-deactivatedaccount/drizzle-automation/fix-with-show-status-flag
Merge into: lp://qastaging/drizzle-automation
Diff against target: 114 lines (+26/-31)
5 files modified
drizzle/automation/client/drizzledb.py (+1/-1)
drizzle/automation/lib/db.py (+1/-1)
drizzle/automation/lib/options.py (+1/-1)
drizzle/automation/lib/util.py (+22/-0)
drizzle/automation/sysbench/run.py (+1/-28)
To merge this branch: bzr merge lp://qastaging/~kalebral-deactivatedaccount/drizzle-automation/fix-with-show-status-flag
Reviewer Review Type Date Requested Status
Jay Pipes Pending
Review via email: mp+18116@code.qastaging.launchpad.net

This proposal supersedes a proposal from 2010-01-26.

To post a comment you must log in.
Revision history for this message
Lee Bieber (kalebral-deactivatedaccount) wrote : Posted in a previous version of this proposal

Fix --with-show-status flag, was not processing output of SHOW GLOBAL STATUS CORRECTLY, also sync'd table name and fields to be consistent with documentation and code. Also made this routine common so that other benchmarks can use it if they want

Revision history for this message
Jay Pipes (jaypipes) wrote : Posted in a previous version of this proposal

Hi!

Couple things I'm having a little trouble with...

1)

21 -CREATE TABLE sysbench_run_show_status (
22 +CREATE TABLE bench_show_status_history (
23 run_id INT NOT NULL
24 +, server_name VARCHAR(255) NOT NULL
25 +, bench_config_name VARCHAR(255) NOT NULL
26 +, run_date DATETIME NOT NULL
27 +, bzr_branch VARCHAR(255) NOT NULL
28 +, bzr_revision INT NOT NULL

The above seems like it is not normalized. The run_id should tie to the run information in the bench_runs table, which makes the bench_config_name, server_name, run_date, bzr_branch, and bzr_revision redundant, since that information is in bench_runs.

2) You've got two log_show_status() functions, one in lib/util.py and one in lib/options.py. Remove one of them and correct the one you keep to not store the redundant columns per #1 above :)

Cheers!

Jay

review: Needs Fixing
Revision history for this message
Lee Bieber (kalebral-deactivatedaccount) wrote : Posted in a previous version of this proposal

per comments from Jay, normalize the --with-show-status table, remove copy paste error

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'drizzle/automation/client/drizzledb.py'
--- drizzle/automation/client/drizzledb.py 2009-06-21 07:52:24 +0000
+++ drizzle/automation/client/drizzledb.py 2010-01-27 05:06:14 +0000
@@ -73,7 +73,7 @@
73 results= []73 results= []
74 # The first line is the field names separated by a tab74 # The first line is the field names separated by a tab
75 # Every line after is a tab-delimited field values.75 # Every line after is a tab-delimited field values.
76 lines= results.split("\n")76 lines= output.split("\n")
77 fields= lines[0].split("\t")77 fields= lines[0].split("\t")
78 for line in lines[1:]:78 for line in lines[1:]:
79 row= {}79 row= {}
8080
=== modified file 'drizzle/automation/lib/db.py'
--- drizzle/automation/lib/db.py 2010-01-21 23:11:44 +0000
+++ drizzle/automation/lib/db.py 2010-01-27 05:06:14 +0000
@@ -174,7 +174,7 @@
174, PRIMARY KEY (run_id, concurrency, iteration)174, PRIMARY KEY (run_id, concurrency, iteration)
175) ENGINE=InnoDB;175) ENGINE=InnoDB;
176176
177CREATE TABLE sysbench_run_show_status (177CREATE TABLE bench_show_status_history (
178 run_id INT NOT NULL178 run_id INT NOT NULL
179, concurrency INT NOT NULL179, concurrency INT NOT NULL
180, variable_name VARCHAR(255) NOT NULL180, variable_name VARCHAR(255) NOT NULL
181181
=== modified file 'drizzle/automation/lib/options.py'
--- drizzle/automation/lib/options.py 2010-01-19 17:14:08 +0000
+++ drizzle/automation/lib/options.py 2010-01-27 05:06:14 +0000
@@ -154,7 +154,7 @@
154 "--with-show-status"154 "--with-show-status"
155 , action="store_true"155 , action="store_true"
156 , default= False156 , default= False
157 , help="SYSBENCH command only. Logs output of SHOW STATUS after last iteration in each concurrency level. [default: %default]"157 , help="DBT2/DRIZZLESLAP/SQLBENCH/SYSBENCH command only. Logs output of SHOW STATUS for each concurrency level. [default: %default]"
158 )158 )
159159
160# Add all the BZR-mode-specific options160# Add all the BZR-mode-specific options
161161
=== modified file 'drizzle/automation/lib/util.py'
--- drizzle/automation/lib/util.py 2010-01-21 23:11:44 +0000
+++ drizzle/automation/lib/util.py 2010-01-27 05:06:14 +0000
@@ -210,3 +210,25 @@
210 bench_config_variables[sec][k]= v210 bench_config_variables[sec][k]= v
211211
212 return bench_config_variables212 return bench_config_variables
213
214def log_show_status(client, run_id, concurrency):
215 from drizzle.automation.lib import db
216
217 logging.info("Logging status variables to database for concurrency %d" % (concurrency))
218
219 results= client.fetchAllAssoc("SHOW GLOBAL STATUS")
220 for result in results:
221 # Strip out non-integer status counters...
222 try:
223 value= int(result['Value'])
224 except:
225 continue
226 name= result['Variable_name']
227 # Write results to the DB
228 sql= """INSERT INTO bench_show_status_history (
229 run_id
230 , concurrency
231 , variable_name
232 , value
233 ) VALUES (%d, %d, '%s', %d)""" % (int(run_id), int(concurrency), name, int(value))
234 db.execute_sql(sql)
213235
=== modified file 'drizzle/automation/sysbench/run.py'
--- drizzle/automation/sysbench/run.py 2010-01-11 06:17:26 +0000
+++ drizzle/automation/sysbench/run.py 2010-01-27 05:06:14 +0000
@@ -37,33 +37,6 @@
37import time37import time
38import datetime38import datetime
3939
40# Store SHOW STATUS output for a concurrency level to database
41def log_show_status(client, run_id, concurrency):
42
43 from drizzle.automation.lib import db
44
45 logging.info("Logging status variables to database for concurrency %d" % (concurrency))
46
47 results= client.fetchAllAssoc("SHOW GLOBAL STATUS")
48 for result in results:
49 # Strip out non-integer status counters...
50 try:
51 value= int(result['Value'])
52 name= result['Variable_name']
53 # Write results to the DB
54 sql= """INSERT INTO sysbench_show_status_history (
55 branch_name
56 , sysbench_configuration
57 , run_date
58 , revno
59 , concurrency
60 , variable_name
61 , value
62 ) VALUES ('%s', '%s', '%s', %d, %d, '%s', %d)""" % (server_version, bench_config_name, run_date, int(concurrency), name, value)
63 db.execute_sql(sql)
64 except:
65 continue
66
67# Stores data for a single sysbench iteration in the database40# Stores data for a single sysbench iteration in the database
68def log_sysbench_iteration(run_id, concurrency, iteration, output):41def log_sysbench_iteration(run_id, concurrency, iteration, output):
6942
@@ -375,7 +348,7 @@
375348
376 # If we store status counters, do so now...349 # If we store status counters, do so now...
377 if variables['with_show_status'] is True and variables['no_store_db'] is False:350 if variables['with_show_status'] is True and variables['no_store_db'] is False:
378 log_show_status(client, run_id, concurrency)351 util.log_show_status(client, run_id, concurrency)
379352
380 server.stop()353 server.stop()
381354

Subscribers

People subscribed via source and target branches

to all changes: