Module Expr Str Import alias Import alias Import alias Import alias ImportFrom alias Import alias Assign NameStore Call AttributeLoad NameLoad attr Str ClassDef bases NameLoad body FunctionDef arguments args NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam NameParam defaults kwarg body Expr Call AttributeLoad NameLoad attr NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad NameLoad Assign AttributeStore NameLoad attr ListLoad Assign AttributeStore NameLoad attr ListLoad Assign AttributeStore NameLoad attr NameLoad For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body Expr Call NameLoad NameLoad NameLoad Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr ListLoad Assign AttributeStore NameLoad attr Str Assign AttributeStore NameLoad attr Str Assign AttributeStore NameLoad attr Str Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr Dict For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body Expr Call NameLoad NameLoad NameLoad Call AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr Dict Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str Str decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str For TupleStore NameStore NameStore Call AttributeLoad NameLoad attr body If CompareIn NameLoad AttributeLoad NameLoad attr body Assign NameStore SubscriptLoad AttributeLoad NameLoad attr Index NameLoad orelse Assign NameStore Call AttributeLoad NameLoad attr NameLoad Assign SubscriptStore AttributeLoad NameLoad attr Index NameLoad ListLoad Expr Call AttributeLoad SubscriptLoad AttributeLoad NameLoad attr Index NameLoad attr BinOpAdd BinOpAdd AttributeLoad NameLoad attr Str NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Num Assign AttributeStore NameLoad attr BinOpAdd BinOpAdd AttributeLoad NameLoad attr Str AttributeLoad NameLoad attr Assign AttributeStore NameLoad attr NameStore Call AttributeLoad AttributeLoad NameLoad attr attr AttributeLoad NameLoad attr NameLoad If Call AttributeLoad NameLoad attr AttributeLoad NameLoad attr body Assign AttributeStore NameLoad attr NameLoad orelse Assign AttributeStore NameLoad attr NameLoad Assign NameStore Call NameLoad SubscriptLoad NameLoad Index Num Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index Num attr Str Assign NameStore BinOpAdd BinOpMult Call NameLoad SubscriptLoad NameLoad Index Num Num Call NameLoad SubscriptLoad Call AttributeLoad SubscriptLoad NameLoad Index Num attr Str Index Num Assign NameStore BinOpAdd BinOpMult BinOpMult NameLoad Num Num NameLoad Assign NameStore Dict Assign SubscriptStore NameLoad Index Str Call NameLoad NameLoad Assign SubscriptStore NameLoad Index Str SubscriptLoad NameLoad Index Num Assign SubscriptStore NameLoad Index Str SubscriptLoad SubscriptLoad NameLoad Index Num Slice Num Assign SubscriptStore NameLoad Index Str SubscriptLoad SubscriptLoad NameLoad Index Num Slice Num Assign SubscriptStore NameLoad Index Str SubscriptLoad NameLoad Index Num Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore SubscriptLoad NameLoad Slice Num Assign NameStore BinOpDiv Call NameLoad NameLoad Num Assign NameStore Dict For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index BinOpAdd BinOpMult Num NameLoad Num attr Str Assign NameStore SubscriptLoad NameLoad Index BinOpMult Num NameLoad Assign SubscriptStore NameLoad Index BinOpAdd Str NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Dict For NameStore SubscriptLoad NameLoad Slice Num body Assign TupleStore NameStore NameStore Call AttributeLoad NameLoad attr Str Assign SubscriptStore NameLoad Index BinOpAdd Str Call AttributeLoad NameLoad attr Str NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Num If BoolOpOr CompareEq SubscriptLoad NameLoad Index Num Str CompareEq SubscriptLoad NameLoad Index Num Str body Assign NameStore Call NameLoad SubscriptLoad NameLoad Slice Num orelse If BoolOpOr CompareEq SubscriptLoad NameLoad Index Num Str CompareEq SubscriptLoad NameLoad Index Num Str body Assign NameStore BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice Num Num orelse If BoolOpOr CompareEq SubscriptLoad NameLoad Index Num Str CompareEq SubscriptLoad NameLoad Index Num Str body Assign NameStore BinOpDiv BinOpDiv Call NameLoad SubscriptLoad NameLoad Slice Num Num Num orelse Assign NameStore BinOpDiv BinOpDiv BinOpDiv Call NameLoad NameLoad Num Num Num Return Call NameLoad NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore SubscriptLoad NameLoad Slice Num Assign NameStore BinOpDiv Call NameLoad NameLoad Num Assign NameStore Dict For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index BinOpAdd BinOpMult Num NameLoad Num attr Str Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index BinOpMult Num NameLoad Assign SubscriptStore NameLoad Index BinOpAdd Str NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore SubscriptLoad NameLoad Slice Num Assign NameStore BinOpDiv Call NameLoad NameLoad Num Assign NameStore Dict For NameStore Call NameLoad NameLoad body Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index BinOpAdd BinOpMult Num NameLoad Num attr Str Assign NameStore Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index BinOpMult Num NameLoad Assign SubscriptStore NameLoad Index BinOpAdd Str NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam NameParam defaults body Expr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore SubscriptLoad NameLoad Index NameLoad Assign NameStore SubscriptLoad NameLoad Index NameLoad If BoolOpOr CompareIn NameLoad AttributeLoad NameLoad attr CompareIn NameLoad AttributeLoad NameLoad attr body Assign NameStore SubscriptLoad Call AttributeLoad NameLoad attr Str Index Num Assign NameStore Dict For NameStore AttributeLoad NameLoad attr body Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If CompareIn NameLoad ListLoad Str Str Str body Assign SubscriptStore NameLoad Index BinOpAdd BinOpAdd BinOpAdd BinOpAdd NameLoad Str NameLoad Str NameLoad Call AttributeLoad NameLoad attr SubscriptLoad NameLoad Index NameLoad orelse If CompareIn NameLoad ListLoad Str Str Str Str body Assign SubscriptStore NameLoad Index BinOpAdd BinOpAdd BinOpAdd BinOpAdd NameLoad Str NameLoad Str Call AttributeLoad NameLoad attr Str SubscriptLoad NameLoad Index NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr Str Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index NameLoad attr Str Assign NameStore BinOpAdd BinOpMult Call NameLoad SubscriptLoad NameLoad Index Num Num Call NameLoad SubscriptLoad NameLoad Index Num Assign SubscriptStore NameLoad Index BinOpAdd BinOpAdd BinOpAdd BinOpAdd NameLoad Str NameLoad Str Str Call NameLoad NameLoad Expr Call AttributeLoad NameLoad attr NameLoad decorator_list FunctionDef arguments args NameParam defaults body Expr Str For NameStore AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr Str NameLoad Assign NameStore NameLoad Assign NameStore Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad If UnaryOpNot NameLoad body Return NameLoad With Call NameLoad NameLoad NameStore body For NameStore NameLoad body Assign NameStore Call AttributeLoad NameLoad attr If UnaryOpNot NameLoad body Continue If Call AttributeLoad NameLoad attr Str NameLoad body Assign AttributeStore NameLoad attr SubscriptLoad NameLoad Index Num Continue Assign NameStore Call AttributeLoad SubscriptLoad NameLoad Index Num attr If CompareEq NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad orelse If AttributeLoad NameLoad attr body If CompareEq NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Str body Expr Call AttributeLoad NameLoad attr NameLoad orelse If CompareEq NameLoad Str body Assign AttributeStore NameLoad attr NameLoad Assign AttributeStore NameLoad attr NameLoad orelse If BoolOpAnd AttributeLoad NameLoad attr CompareGtE Call NameLoad NameLoad Call NameLoad AttributeLoad NameLoad attr body Expr Call AttributeLoad NameLoad attr NameLoad For NameStore Call AttributeLoad AttributeLoad NameLoad attr attr body Expr Call AttributeLoad AttributeLoad NameLoad attr attr NameLoad With Call NameLoad NameLoad Str NameStore body Expr Call AttributeLoad NameLoad attr Call AttributeLoad Str attr SubscriptLoad AttributeLoad NameLoad attr Index NameLoad Expr Call AttributeLoad NameLoad attr Return NameLoad decorator_list decorator_list
N/A N/A 
Copyright 2013 LinkedIn Corp. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
 N/A gc N/A os N/A re N/A logging naarad.metrics.metric Metric N/A naarad.utils N/A logger N/A N/A logging getLogger naarad.metrics.top_metric TopMetric N/A Metric N/A __init__ N/A N/A self metric_type infile hostname aggr_metrics output_directory resource_path label ts_start ts_end rule_strings important_sub_metrics anomaly_detection_metrics N/A other_options N/A N/A N/A N/A Metric __init__ self metric_type infile hostname aggr_metrics output_directory resource_path label ts_start ts_end rule_strings important_sub_metrics anomaly_detection_metrics N/A N/A self PID N/A N/A N/A self COMMAND N/A N/A N/A self ts_valid_lines True N/A N/A key val N/A N/A other_options iteritems N/A N/A N/A setattr self key N/A N/A val split N/A N/A self sub_metrics None N/A N/A self process_headers N/A N/A N/A self ts N/A N/A self ts_date N/A N/A self ts_time N/A N/A self saw_pid False N/A N/A self data N/A N/A N/A key val N/A N/A other_options iteritems N/A N/A N/A setattr self key N/A N/A val split N/A N/A self sub_metric_description N/A uptime_minute num_users load_aver_1_minute load_aver_5_minute load_aver_15_minute tasks_total tasks_running tasks_sleeping tasks_stopped tasks_zombie cpu_us cpu_sy cpu_id cpu_ni cpu_wa cpu_hi cpu_si cpu_st mem_total mem_used mem_free mem_buffers swap_total swap_used swap_free swap_cached uptime of the machine users sessions logged in average load on the system (last 1 minute) average load on the system (last 5 minutes) average load on the system (last 15 minutes) total processes processes running processes sleeping processes stopped zombies cpu percentage of running user processes cpu percentage of running system processes cpu percentage of idel time cpu percentage of running niced processes cpu percentage of waiting for IO cpu percentage of serving hardware IRQ cpu percentage of serving software IRQ cpu percentage of being stolen total memory in GB total memory in use in GB total free memory in GB total buffers in GB total swap size in GB total swap in use in GB total free swap in GB total swap cache in GB N/A put_values_into_data N/A N/A self values N/A N/A N/A 
    Take the (col, value) in 'values', append value into 'col' in self.data[]
     N/A N/A col value N/A N/A values items N/A N/A N/A col N/A self column_csv_map N/A N/A out_csv N/A N/A self column_csv_map N/A col N/A N/A out_csv N/A N/A self get_csv col N/A N/A N/A self data N/A out_csv N/A N/A N/A N/A N/A N/A self data N/A out_csv append N/A N/A N/A self ts , value N/A process_top_line N/A N/A self words N/A N/A N/A 
    Process the line starting with "top"
    Example log:   top - 00:00:02 up 32 days,  7:08, 19 users,  load average: 0.00, 0.00, 0.00
     N/A N/A self ts_time N/A words N/A 2 N/A N/A self ts N/A N/A N/A self ts_date N/A self ts_time N/A N/A self ts ts N/A N/A N/A naarad utils get_standardized_timestamp N/A self ts None N/A N/A N/A self ts_out_of_range N/A self ts N/A N/A N/A self ts_valid_lines False N/A N/A N/A self ts_valid_lines True N/A up_days N/A int N/A words N/A 4 N/A up_hour_minute N/A N/A N/A words N/A 6 split : N/A up_minutes N/A N/A N/A int N/A up_hour_minute N/A 0 60 N/A int N/A N/A N/A N/A up_hour_minute N/A 1 split , N/A 0 N/A uptime_minute N/A N/A N/A up_days 24 60 up_minutes N/A values N/A N/A N/A values N/A uptime_minute N/A str uptime_minute N/A N/A values N/A num_users N/A words N/A 7 N/A N/A values N/A load_aver_1_minute N/A N/A words N/A 11 N/A -1 N/A N/A values N/A load_aver_5_minute N/A N/A words N/A 12 N/A -1 N/A N/A values N/A load_aver_15_minute N/A words N/A 13 N/A N/A N/A self put_values_into_data values N/A process_tasks_line N/A N/A self words N/A N/A N/A 
    Process the line starting with "Tasks:"
    Example log:   Tasks: 446 total,   1 running, 442 sleeping,   2 stopped,   1 zombie
     N/A words N/A words N/A 1 N/A length N/A N/A len words 2 N/A values N/A N/A offset N/A range length N/A N/A k N/A N/A N/A words N/A N/A N/A 2 offset 1 strip , N/A v N/A words N/A N/A 2 offset N/A N/A values N/A N/A tasks_ k v N/A N/A N/A self put_values_into_data values N/A process_cpu_line N/A N/A self words N/A N/A N/A 
    Process the line starting with "Cpu(s):"
    Example log: Cpu(s):  1.3%us,  0.5%sy,  0.0%ni, 98.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
     N/A values N/A N/A word N/A words N/A 1 N/A N/A N/A val key N/A N/A word split % N/A N/A values N/A N/A cpu_ N/A N/A key strip , val N/A N/A N/A self put_values_into_data values N/A convert_to_G N/A N/A self word N/A N/A N/A 
    Given a size such as '2333M', return the converted value in G
     N/A value 0.0 N/A N/A N/A N/A word N/A -1 G N/A N/A word N/A -1 g N/A N/A value N/A float N/A word N/A -1 N/A N/A N/A N/A N/A word N/A -1 M N/A N/A word N/A -1 m N/A N/A value N/A N/A float N/A word N/A -1 1000.0 N/A N/A N/A N/A N/A word N/A -1 K N/A N/A word N/A -1 k N/A N/A value N/A N/A N/A float N/A word N/A -1 1000.0 1000.0 N/A N/A value N/A N/A N/A N/A float word 1000.0 1000.0 1000.0 N/A N/A str value N/A process_mem_line N/A N/A self words N/A N/A N/A 
    Process the line starting with "Mem:"
    Example log: Mem:    62.841G total,   16.038G used,   46.803G free,  650.312M buffers
    For each value, needs to convert to 'G' (needs to handle cases of K, M)
     N/A words N/A words N/A 1 N/A length N/A N/A len words 2 N/A values N/A N/A offset N/A range length N/A N/A k N/A N/A N/A words N/A N/A N/A 2 offset 1 strip , N/A v N/A N/A self convert_to_G N/A words N/A N/A 2 offset N/A N/A values N/A N/A mem_ k v N/A N/A N/A self put_values_into_data values N/A process_swap_line N/A N/A self words N/A N/A N/A 
    Process the line starting with "Swap:"
    Example log: Swap:   63.998G total,    0.000k used,   63.998G free,   11.324G cached
    For each value, needs to convert to 'G' (needs to handle cases of K, M)
     N/A words N/A words N/A 1 N/A length N/A N/A len words 2 N/A values N/A N/A offset N/A range length N/A N/A k N/A N/A N/A words N/A N/A N/A 2 offset 1 strip , N/A v N/A N/A self convert_to_G N/A words N/A N/A 2 offset N/A N/A values N/A N/A swap_ k v N/A N/A N/A self put_values_into_data values N/A process_individual_command N/A N/A self words N/A N/A N/A 
    process the individual lines like this:
    #PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    29303 root      20   0 35300 2580 1664 R  3.9  0.0   0:00.02 top
    11 root      RT   0     0    0    0 S  1.9  0.0   0:18.87 migration/2
    3702 root      20   0 34884 4192 1692 S  1.9  0.0  31:40.47 cf-serverd
    It does not record all processes due to memory concern; rather only records interested processes (based on user input of PID and COMMAND)
     N/A pid_index N/A N/A N/A self process_headers index PID N/A proces_index N/A N/A N/A self process_headers index COMMAND N/A pid N/A words N/A pid_index N/A process N/A words N/A proces_index N/A N/A N/A pid N/A self PID N/A process N/A self COMMAND N/A N/A process_name N/A N/A N/A process split / N/A 0 N/A values N/A N/A word_col N/A self process_headers N/A N/A word_index N/A N/A N/A self process_headers index word_col N/A N/A word_col N/A VIRT RES SHR N/A N/A N/A values N/A N/A N/A N/A N/A process_name _ pid _ word_col N/A N/A self convert_to_G N/A words N/A word_index N/A N/A N/A word_col N/A PR NI %CPU %MEM N/A N/A N/A values N/A N/A N/A N/A N/A process_name _ pid _ N/A N/A word_col strip % N/A words N/A word_index N/A uptime_index N/A N/A N/A self process_headers index TIME+ N/A uptime N/A N/A N/A words N/A uptime_index split : N/A uptime_sec N/A N/A N/A float N/A uptime N/A 0 60 N/A float N/A uptime N/A 1 N/A N/A values N/A N/A N/A N/A N/A process_name _ pid _ TIME N/A str uptime_sec N/A N/A N/A self put_values_into_data values N/A parse N/A N/A self N/A N/A N/A 
    Parse the top output file
    Return status of the metric parse

    The raw log file is like the following:
    2014-06-23
    top - 00:00:02 up 18 days,  7:08, 19 users,  load average: 0.05, 0.03, 0.00
    Tasks: 447 total,   1 running, 443 sleeping,   2 stopped,   1 zombie
    Cpu(s):  1.6%us,  0.5%sy,  0.0%ni, 97.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
    Mem:    62.841G total,   15.167G used,   47.675G free,  643.434M buffers
    Swap:   63.998G total,    0.000k used,   63.998G free,   11.324G cached

    PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1730 root      20   0 4457m  10m 3328 S  1.9  0.0  80:13.45 lwregd
    The log lines can be generated by echo $t >> $RESULT/top.out &; top -b -n $COUNT -d $INTERVAL | grep -A 40 '^top' >> $RESULT/top.out &
     N/A infile N/A self infile_list N/A N/A N/A N/A logger info Processing : %s infile N/A status True N/A file_status N/A N/A N/A naarad utils is_valid_file infile N/A N/A file_status N/A N/A False N/A N/A open infile fh N/A N/A line fh N/A N/A words N/A N/A line split N/A N/A words N/A N/A N/A N/A N/A re match ^\d\d\d\d-\d\d-\d\d$ line N/A N/A N/A self ts_date N/A words N/A 0 N/A N/A prefix_word N/A N/A N/A words N/A 0 strip N/A N/A prefix_word top N/A N/A N/A N/A self process_top_line words N/A N/A self saw_pid False N/A N/A N/A self ts_valid_lines N/A N/A N/A prefix_word Tasks: N/A N/A N/A N/A self process_tasks_line words N/A N/A N/A prefix_word Cpu(s): N/A N/A N/A N/A self process_cpu_line words N/A N/A N/A prefix_word Mem: N/A N/A N/A N/A self process_mem_line words N/A N/A N/A prefix_word Swap: N/A N/A N/A N/A self process_swap_line words N/A N/A N/A prefix_word PID N/A N/A N/A self saw_pid True N/A N/A self process_headers words N/A N/A N/A N/A self saw_pid N/A N/A len words N/A len N/A self process_headers N/A N/A N/A N/A self process_individual_command words N/A out_csv N/A N/A N/A self data keys N/A N/A N/A N/A N/A self csv_files append out_csv N/A N/A open out_csv w fh N/A N/A N/A N/A fh write N/A N/A join N/A N/A self data N/A out_csv N/A N/A N/A gc collect N/A status N/A N/A