Drizzle Automation

Merge lp://qastaging/~kalebral-deactivatedaccount/drizzle-automation/fix-with-show-status-flag into lp://qastaging/drizzle-automation

  1. fix-with-show-status-flag
  2. Merge into trunk
Proposed by Lee Bieber
Status: Merged
Merged at revision: not available
Proposed branch: lp://qastaging/~kalebral-deactivatedaccount/drizzle-automation/fix-with-show-status-flag
Merge into: lp://qastaging/drizzle-automation
Diff against target: 114 lines (+26/-31)
5 files modified
drizzle/automation/client/drizzledb.py (+1/-1)
drizzle/automation/lib/db.py (+1/-1)
drizzle/automation/lib/options.py (+1/-1)
drizzle/automation/lib/util.py (+22/-0)
drizzle/automation/sysbench/run.py (+1/-28)
To merge this branch: bzr merge lp://qastaging/~kalebral-deactivatedaccount/drizzle-automation/fix-with-show-status-flag
Reviewer Review Type Date Requested Status
Jay Pipes Pending
Review via email: mp+18116@code.qastaging.launchpad.net

This proposal supersedes a proposal from 2010-01-26.

To post a comment you must log in.
Revision history for this message
Lee Bieber (kalebral-deactivatedaccount) wrote : Posted in a previous version of this proposal

Fix --with-show-status flag, was not processing output of SHOW GLOBAL STATUS CORRECTLY, also sync'd table name and fields to be consistent with documentation and code. Also made this routine common so that other benchmarks can use it if they want

Revision history for this message
Jay Pipes (jaypipes) wrote : Posted in a previous version of this proposal

Hi!

Couple things I'm having a little trouble with...

1)

21 -CREATE TABLE sysbench_run_show_status (
22 +CREATE TABLE bench_show_status_history (
23 run_id INT NOT NULL
24 +, server_name VARCHAR(255) NOT NULL
25 +, bench_config_name VARCHAR(255) NOT NULL
26 +, run_date DATETIME NOT NULL
27 +, bzr_branch VARCHAR(255) NOT NULL
28 +, bzr_revision INT NOT NULL

The above seems like it is not normalized. The run_id should tie to the run information in the bench_runs table, which makes the bench_config_name, server_name, run_date, bzr_branch, and bzr_revision redundant, since that information is in bench_runs.

2) You've got two log_show_status() functions, one in lib/util.py and one in lib/options.py. Remove one of them and correct the one you keep to not store the redundant columns per #1 above :)

Cheers!

Jay

review: Needs Fixing
Revision history for this message
Lee Bieber (kalebral-deactivatedaccount) wrote : Posted in a previous version of this proposal

per comments from Jay, normalize the --with-show-status table, remove copy paste error

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'drizzle/automation/client/drizzledb.py'
2--- drizzle/automation/client/drizzledb.py 2009-06-21 07:52:24 +0000
3+++ drizzle/automation/client/drizzledb.py 2010-01-27 05:06:14 +0000
4@@ -73,7 +73,7 @@
5 results= []
6 # The first line is the field names separated by a tab
7 # Every line after is a tab-delimited field values.
8- lines= results.split("\n")
9+ lines= output.split("\n")
10 fields= lines[0].split("\t")
11 for line in lines[1:]:
12 row= {}
13
14=== modified file 'drizzle/automation/lib/db.py'
15--- drizzle/automation/lib/db.py 2010-01-21 23:11:44 +0000
16+++ drizzle/automation/lib/db.py 2010-01-27 05:06:14 +0000
17@@ -174,7 +174,7 @@
18 , PRIMARY KEY (run_id, concurrency, iteration)
19 ) ENGINE=InnoDB;
20
21-CREATE TABLE sysbench_run_show_status (
22+CREATE TABLE bench_show_status_history (
23 run_id INT NOT NULL
24 , concurrency INT NOT NULL
25 , variable_name VARCHAR(255) NOT NULL
26
27=== modified file 'drizzle/automation/lib/options.py'
28--- drizzle/automation/lib/options.py 2010-01-19 17:14:08 +0000
29+++ drizzle/automation/lib/options.py 2010-01-27 05:06:14 +0000
30@@ -154,7 +154,7 @@
31 "--with-show-status"
32 , action="store_true"
33 , default= False
34- , help="SYSBENCH command only. Logs output of SHOW STATUS after last iteration in each concurrency level. [default: %default]"
35+ , help="DBT2/DRIZZLESLAP/SQLBENCH/SYSBENCH command only. Logs output of SHOW STATUS for each concurrency level. [default: %default]"
36 )
37
38 # Add all the BZR-mode-specific options
39
40=== modified file 'drizzle/automation/lib/util.py'
41--- drizzle/automation/lib/util.py 2010-01-21 23:11:44 +0000
42+++ drizzle/automation/lib/util.py 2010-01-27 05:06:14 +0000
43@@ -210,3 +210,25 @@
44 bench_config_variables[sec][k]= v
45
46 return bench_config_variables
47+
48+def log_show_status(client, run_id, concurrency):
49+ from drizzle.automation.lib import db
50+
51+ logging.info("Logging status variables to database for concurrency %d" % (concurrency))
52+
53+ results= client.fetchAllAssoc("SHOW GLOBAL STATUS")
54+ for result in results:
55+ # Strip out non-integer status counters...
56+ try:
57+ value= int(result['Value'])
58+ except:
59+ continue
60+ name= result['Variable_name']
61+ # Write results to the DB
62+ sql= """INSERT INTO bench_show_status_history (
63+ run_id
64+ , concurrency
65+ , variable_name
66+ , value
67+ ) VALUES (%d, %d, '%s', %d)""" % (int(run_id), int(concurrency), name, int(value))
68+ db.execute_sql(sql)
69
70=== modified file 'drizzle/automation/sysbench/run.py'
71--- drizzle/automation/sysbench/run.py 2010-01-11 06:17:26 +0000
72+++ drizzle/automation/sysbench/run.py 2010-01-27 05:06:14 +0000
73@@ -37,33 +37,6 @@
74 import time
75 import datetime
76
77-# Store SHOW STATUS output for a concurrency level to database
78-def log_show_status(client, run_id, concurrency):
79-
80- from drizzle.automation.lib import db
81-
82- logging.info("Logging status variables to database for concurrency %d" % (concurrency))
83-
84- results= client.fetchAllAssoc("SHOW GLOBAL STATUS")
85- for result in results:
86- # Strip out non-integer status counters...
87- try:
88- value= int(result['Value'])
89- name= result['Variable_name']
90- # Write results to the DB
91- sql= """INSERT INTO sysbench_show_status_history (
92- branch_name
93- , sysbench_configuration
94- , run_date
95- , revno
96- , concurrency
97- , variable_name
98- , value
99- ) VALUES ('%s', '%s', '%s', %d, %d, '%s', %d)""" % (server_version, bench_config_name, run_date, int(concurrency), name, value)
100- db.execute_sql(sql)
101- except:
102- continue
103-
104 # Stores data for a single sysbench iteration in the database
105 def log_sysbench_iteration(run_id, concurrency, iteration, output):
106
107@@ -375,7 +348,7 @@
108
109 # If we store status counters, do so now...
110 if variables['with_show_status'] is True and variables['no_store_db'] is False:
111- log_show_status(client, run_id, concurrency)
112+ util.log_show_status(client, run_id, concurrency)
113
114 server.stop()
115

Subscribers

People subscribed via source and target branches

to all changes: