← Index
NYTProf Performance Profile   « line view »
For /usr/local/bin/sa-learn
  Run on Tue Nov 7 05:38:10 2017
Reported on Tue Nov 7 06:16:06 2017

Filename/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/DBBasedAddrList.pm
StatementsExecuted 46644972 statements in 1470s
Subroutines
Calls P F Exclusive
Time		 Inclusive
Time		 Subroutine
43831963s1469sMail::SpamAssassin::DBBasedAddrList::::remove_entryMail::SpamAssassin::DBBasedAddrList::remove_entry
232983101188.1s88.1sMail::SpamAssassin::DBBasedAddrList::::CORE:regcompMail::SpamAssassin::DBBasedAddrList::CORE:regcomp (opcode)
232987482146.4s46.4sMail::SpamAssassin::DBBasedAddrList::::CORE:matchMail::SpamAssassin::DBBasedAddrList::CORE:match (opcode)
311831254ms764msMail::SpamAssassin::DBBasedAddrList::::get_addr_entryMail::SpamAssassin::DBBasedAddrList::get_addr_entry
289831210ms315msMail::SpamAssassin::DBBasedAddrList::::add_scoreMail::SpamAssassin::DBBasedAddrList::add_score
1111.03ms1.38msMail::SpamAssassin::DBBasedAddrList::::BEGIN@26Mail::SpamAssassin::DBBasedAddrList::BEGIN@26
111242µs708msMail::SpamAssassin::DBBasedAddrList::::new_checkerMail::SpamAssassin::DBBasedAddrList::new_checker
111147µs5.48sMail::SpamAssassin::DBBasedAddrList::::finishMail::SpamAssassin::DBBasedAddrList::finish
22166µs66µsMail::SpamAssassin::DBBasedAddrList::::CORE:umaskMail::SpamAssassin::DBBasedAddrList::CORE:umask (opcode)
11156µs68µsMail::SpamAssassin::DBBasedAddrList::::BEGIN@20Mail::SpamAssassin::DBBasedAddrList::BEGIN@20
11152µs69µsMail::SpamAssassin::DBBasedAddrList::::newMail::SpamAssassin::DBBasedAddrList::new
11148µs114µsMail::SpamAssassin::DBBasedAddrList::::BEGIN@23Mail::SpamAssassin::DBBasedAddrList::BEGIN@23
11134µs196µsMail::SpamAssassin::DBBasedAddrList::::BEGIN@27Mail::SpamAssassin::DBBasedAddrList::BEGIN@27
11134µs263µsMail::SpamAssassin::DBBasedAddrList::::BEGIN@28Mail::SpamAssassin::DBBasedAddrList::BEGIN@28
11133µs2.30msMail::SpamAssassin::DBBasedAddrList::::BEGIN@24Mail::SpamAssassin::DBBasedAddrList::BEGIN@24
11127µs34µsMail::SpamAssassin::DBBasedAddrList::::BEGIN@22Mail::SpamAssassin::DBBasedAddrList::BEGIN@22
11124µs64µsMail::SpamAssassin::DBBasedAddrList::::BEGIN@21Mail::SpamAssassin::DBBasedAddrList::BEGIN@21
Call graph for these subroutines as a Graphviz dot language file.
Line State
ments		 Time
on line		 Calls Time
in subs		 Code
1# <@LICENSE>
2# Licensed to the Apache Software Foundation (ASF) under one or more
3# contributor license agreements. See the NOTICE file distributed with
4# this work for additional information regarding copyright ownership.
5# The ASF licenses this file to you under the Apache License, Version 2.0
6# (the "License"); you may not use this file except in compliance with
7# the License. You may obtain a copy of the License at:
8#
9# http://www.apache.org/licenses/LICENSE-2.0
10#
11# Unless required by applicable law or agreed to in writing, software
12# distributed under the License is distributed on an "AS IS" BASIS,
13# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14# See the License for the specific language governing permissions and
15# limitations under the License.
16# </@LICENSE>
17
18package Mail::SpamAssassin::DBBasedAddrList;
19
20271µs280µs
# spent 68µs (56+12) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@20 which was called: # once (56µs+12µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 20
use strict;
# spent 68µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@20 # spent 12µs making 1 call to strict::import
21282µs2104µs
# spent 64µs (24+40) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@21 which was called: # once (24µs+40µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 21
use warnings;
# spent 64µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@21 # spent 40µs making 1 call to warnings::import
22285µs241µs
# spent 34µs (27+7) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@22 which was called: # once (27µs+7µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 22
use bytes;
# spent 34µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@22 # spent 7µs making 1 call to bytes::import
23287µs2179µs
# spent 114µs (48+66) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@23 which was called: # once (48µs+66µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 23
use re 'taint';
# spent 114µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@23 # spent 66µs making 1 call to re::import
24283µs24.56ms
# spent 2.30ms (33µs+2.26) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@24 which was called: # once (33µs+2.26ms) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 24
use Fcntl;
# spent 2.30ms making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@24 # spent 2.26ms making 1 call to Exporter::import
25
262381µs11.38ms
# spent 1.38ms (1.03+347µs) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@26 which was called: # once (1.03ms+347µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 26
use Mail::SpamAssassin::PersistentAddrList;
# spent 1.38ms making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@26
27281µs2357µs
# spent 196µs (34+162) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@27 which was called: # once (34µs+162µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 27
use Mail::SpamAssassin::Util qw(untaint_var);
# spent 196µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@27 # spent 162µs making 1 call to Exporter::import
2821.56ms2491µs
# spent 263µs (34+229) within Mail::SpamAssassin::DBBasedAddrList::BEGIN@28 which was called: # once (34µs+229µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 28
use Mail::SpamAssassin::Logger;
# spent 263µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::BEGIN@28 # spent 229µs making 1 call to Exporter::import
29
30113µsour @ISA = qw(Mail::SpamAssassin::PersistentAddrList);
31
32###########################################################################
33
34
# spent 69µs (52+17) within Mail::SpamAssassin::DBBasedAddrList::new which was called: # once (52µs+17µs) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 2 of (eval 1135)[Mail/SpamAssassin/Plugin/TxRep.pm:1645]
sub new {
3512µs my $class = shift;
3612µs $class = ref($class) || $class;
37118µs117µs my $self = $class->SUPER::new(@_);
# spent 17µs making 1 call to Mail::SpamAssassin::PersistentAddrList::new
38113µs $self->{class} = $class;
3912µs bless ($self, $class);
40114µs $self;
41}
42
43###########################################################################
44
45
# spent 708ms (242µs+707) within Mail::SpamAssassin::DBBasedAddrList::new_checker which was called: # once (242µs+707ms) by Mail::SpamAssassin::Plugin::TxRep::open_storages at line 1654 of Mail/SpamAssassin/Plugin/TxRep.pm
sub new_checker {
4612µs my ($factory, $main) = @_;
4712µs my $class = $factory->{class};
48
49112µs my $self = {
50 'main' => $main,
51 'accum' => { },
52 'is_locked' => 0,
53 'locked_file' => ''
54 };
55
56110µs my @order = split (' ', $main->{conf}->{auto_whitelist_db_modules});
5717µs1195µs untaint_var(\@order);
# spent 195µs making 1 call to Mail::SpamAssassin::Util::untaint_var
58116µs180µs my $dbm_module = Mail::SpamAssassin::Util::first_available_module (@order);
# spent 80µs making 1 call to Mail::SpamAssassin::Util::first_available_module
5912µs if (!$dbm_module) {
60 die "auto-whitelist: cannot find a usable DB package from auto_whitelist_db_modules: " .
61 $main->{conf}->{auto_whitelist_db_modules}."\n";
62 }
63
64190µs162µs my $umask = umask ~ (oct($main->{conf}->{auto_whitelist_file_mode}));
# spent 62µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::CORE:umask
65
66 # if undef then don't worry -- empty hash!
6714µs if (defined($main->{conf}->{auto_whitelist_path})) {
68110µs1582µs my $path = $main->sed_path($main->{conf}->{auto_whitelist_path});
# spent 582µs making 1 call to Mail::SpamAssassin::sed_path
6912µs my ($mod1, $mod2);
70
71116µs1705ms if ($main->{locker}->safe_lock
# spent 705ms making 1 call to Mail::SpamAssassin::Locker::UnixNFSSafe::safe_lock
72 ($path, 30, $main->{conf}->{auto_whitelist_file_mode}))
73 {
7416µs $self->{locked_file} = $path;
7514µs $self->{is_locked} = 1;
7614µs ($mod1, $mod2) = ('R/W', O_RDWR | O_CREAT);
77 }
78 else {
79 $self->{is_locked} = 0;
80 ($mod1, $mod2) = ('R/O', O_RDONLY);
81 }
82
83111µs17µs dbg("auto-whitelist: tie-ing to DB file of type $dbm_module $mod1 in $path");
# spent 7µs making 1 call to Mail::SpamAssassin::Logger::dbg
84
85114µs11.05ms ($self->{is_locked} && $dbm_module eq 'DB_File') and
# spent 1.05ms making 1 call to Mail::SpamAssassin::Util::avoid_db_file_locking_bug
86 Mail::SpamAssassin::Util::avoid_db_file_locking_bug($path);
87
88249µs1242µs if (! tie %{ $self->{accum} }, $dbm_module, $path, $mod2,
# spent 242µs making 1 call to DB_File::TIEHASH
89 oct($main->{conf}->{auto_whitelist_file_mode}) & 0666)
90 {
91 my $err = $!; # might get overwritten later
92 if ($self->{is_locked}) {
93 $self->{main}->{locker}->safe_unlock($self->{locked_file});
94 $self->{is_locked} = 0;
95 }
96 die "auto-whitelist: cannot open auto_whitelist_path $path: $err\n";
97 }
98 }
99113µs14µs umask $umask;
# spent 4µs making 1 call to Mail::SpamAssassin::DBBasedAddrList::CORE:umask
100
10113µs bless ($self, $class);
102115µs return $self;
103}
104
105###########################################################################
106
107
# spent 5.48s (147µs+5.48) within Mail::SpamAssassin::DBBasedAddrList::finish which was called: # once (147µs+5.48s) by Mail::SpamAssassin::Plugin::TxRep::finish at line 1722 of Mail/SpamAssassin/Plugin/TxRep.pm
sub finish {
10812µs my $self = shift;
10917µs17µs dbg("auto-whitelist: DB addr list: untie-ing and unlocking");
# spent 7µs making 1 call to Mail::SpamAssassin::Logger::dbg
11023.88ms13.82ms untie %{$self->{accum}};
# spent 3.82ms making 1 call to DB_File::DESTROY
111122µs if ($self->{is_locked}) {
112113µs114µs dbg("auto-whitelist: DB addr list: file locked, breaking lock");
# spent 14µs making 1 call to Mail::SpamAssassin::Logger::dbg
113117µs15.47s $self->{main}->{locker}->safe_unlock ($self->{locked_file});
# spent 5.47s making 1 call to Mail::SpamAssassin::Locker::UnixNFSSafe::safe_unlock
11418µs $self->{is_locked} = 0;
115 }
116 # TODO: untrap signals to unlock the db file here
117}
118
119###########################################################################
120
121
# spent 764ms (254+510) within Mail::SpamAssassin::DBBasedAddrList::get_addr_entry which was called 3118 times, avg 245µs/call: # 3114 times (253ms+510ms) by Mail::SpamAssassin::Plugin::TxRep::get_sender at line 1539 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 245µs/call # 2 times (178µs+229µs) by Mail::SpamAssassin::Plugin::TxRep::modify_reputation at line 1608 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 204µs/call # 2 times (155µs+242µs) by Mail::SpamAssassin::Plugin::TxRep::modify_reputation at line 1618 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 198µs/call
sub get_addr_entry {
122311813.9ms my ($self, $addr, $signedby) = @_;
123
124311821.1ms my $entry = {
125 addr => $addr,
126 };
127
1283118309ms3118248ms $entry->{count} = $self->{accum}->{$addr} || 0;
# spent 248ms making 3118 calls to DB_File::FETCH, avg 79µs/call
1293118298ms3118232ms $entry->{totscore} = $self->{accum}->{$addr.'|totscore'} || 0;
# spent 232ms making 3118 calls to DB_File::FETCH, avg 74µs/call
130
131311845.2ms311830.9ms dbg("auto-whitelist: db-based $addr scores ".$entry->{count}.'/'.$entry->{totscore});
# spent 30.9ms making 3118 calls to Mail::SpamAssassin::Logger::dbg, avg 10µs/call
132311836.6ms return $entry;
133}
134
135###########################################################################
136
137
# spent 315ms (210+105) within Mail::SpamAssassin::DBBasedAddrList::add_score which was called 2898 times, avg 109µs/call: # 1615 times (112ms+58.6ms) by Mail::SpamAssassin::Plugin::TxRep::add_score at line 1573 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 106µs/call # 1281 times (98.4ms+46.1ms) by Mail::SpamAssassin::Plugin::TxRep::remove_score at line 1596 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 113µs/call # 2 times (154µs+55µs) by Mail::SpamAssassin::Plugin::TxRep::modify_reputation at line 1619 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 105µs/call
sub add_score {
13828989.39ms my($self, $entry, $score) = @_;
139
14028988.71ms $entry->{count} ||= 0;
14128985.97ms $entry->{addr} ||= '';
142
14328986.19ms $entry->{count}++;
144289810.5ms $entry->{totscore} += $score;
145
146289839.8ms289824.5ms dbg("auto-whitelist: add_score: new count: ".$entry->{count}.", new totscore: ".$entry->{totscore});
# spent 24.5ms making 2898 calls to Mail::SpamAssassin::Logger::dbg, avg 8µs/call
147
148289895.6ms289842.8ms $self->{accum}->{$entry->{addr}} = $entry->{count};
# spent 42.8ms making 2898 calls to DB_File::STORE, avg 15µs/call
149289882.7ms289837.4ms $self->{accum}->{$entry->{addr}.'|totscore'} = $entry->{totscore};
# spent 37.4ms making 2898 calls to DB_File::STORE, avg 13µs/call
150289837.0ms return $entry;
151}
152
153###########################################################################
154
155
# spent 1469s (963+506) within Mail::SpamAssassin::DBBasedAddrList::remove_entry which was called 438 times, avg 3.35s/call: # 218 times (483s+275s) by Mail::SpamAssassin::Plugin::TxRep::check_reputation at line 1509 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 3.48s/call # 218 times (476s+228s) by Mail::SpamAssassin::Plugin::TxRep::check_reputation at line 1500 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 3.23s/call # 2 times (4.31s+1.91s) by Mail::SpamAssassin::Plugin::TxRep::modify_reputation at line 1613 of Mail/SpamAssassin/Plugin/TxRep.pm, avg 3.11s/call
sub remove_entry {
156438968µs my ($self, $entry) = @_;
157
1584381.59ms my $addr = $entry->{addr};
15943815.7ms4388.46ms delete $self->{accum}->{$addr};
# spent 8.46ms making 438 calls to DB_File::DELETE, avg 19µs/call
16043822.0ms4386.67ms delete $self->{accum}->{$addr.'|totscore'};
# spent 6.67ms making 438 calls to DB_File::DELETE, avg 15µs/call
161
16243813.0ms4383.33ms if ($addr =~ /^(.*)\|ip=none$/) {
# spent 3.33ms making 438 calls to Mail::SpamAssassin::DBBasedAddrList::CORE:match, avg 8µs/call
163 # it doesn't have an IP attached.
164 # try to delete any per-IP entries for this addr as well.
165 # could be slow...
1664381.72ms my $mailaddr = $1;
167
16823299186999s46597058371s while (my ($key, $value) = each %{$self->{accum}}) {
# spent 213s making 23298310 calls to DB_File::FETCH, avg 9µs/call # spent 158s making 23298310 calls to DB_File::NEXTKEY, avg 7µs/call # spent 37.4ms making 438 calls to DB_File::FIRSTKEY, avg 85µs/call
169 # regex will catch both key and key|totscore entries and delete them
17023298310470s46596620134s if ($key =~ /^\Q${mailaddr}\E\|/) {
# spent 88.1s making 23298310 calls to Mail::SpamAssassin::DBBasedAddrList::CORE:regcomp, avg 4µs/call # spent 46.4s making 23298310 calls to Mail::SpamAssassin::DBBasedAddrList::CORE:match, avg 2µs/call
1714112µs437µs delete $self->{accum}->{$key};
# spent 37µs making 4 calls to DB_File::DELETE, avg 9µs/call
172 }
173 }
174 }
175}
176
177###########################################################################
178
179110µs1;
 
# spent 46.4s within Mail::SpamAssassin::DBBasedAddrList::CORE:match which was called 23298748 times, avg 2µs/call: # 23298310 times (46.4s+0s) by Mail::SpamAssassin::DBBasedAddrList::remove_entry at line 170, avg 2µs/call # 438 times (3.33ms+0s) by Mail::SpamAssassin::DBBasedAddrList::remove_entry at line 162, avg 8µs/call
sub Mail::SpamAssassin::DBBasedAddrList::CORE:match; # opcode
# spent 88.1s within Mail::SpamAssassin::DBBasedAddrList::CORE:regcomp which was called 23298310 times, avg 4µs/call: # 23298310 times (88.1s+0s) by Mail::SpamAssassin::DBBasedAddrList::remove_entry at line 170, avg 4µs/call
sub Mail::SpamAssassin::DBBasedAddrList::CORE:regcomp; # opcode
# spent 66µs within Mail::SpamAssassin::DBBasedAddrList::CORE:umask which was called 2 times, avg 33µs/call: # once (62µs+0s) by Mail::SpamAssassin::DBBasedAddrList::new_checker at line 64 # once (4µs+0s) by Mail::SpamAssassin::DBBasedAddrList::new_checker at line 99
sub Mail::SpamAssassin::DBBasedAddrList::CORE:umask; # opcode
Report produced by the NYTProf 6.04 Perl profiler, developed by Tim Bunce based on work by Adam Kaplan and Salvador Fandiño García.