Filename | /usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm |
Statements | Executed 176375 statements in 1.40s |
Calls | P | F | Exclusive Time |
Inclusive Time |
Subroutine |
---|---|---|---|---|---|
234 | 1 | 1 | 602ms | 15.0s | parsed_metadata | Mail::SpamAssassin::Plugin::URIDNSBL::
8043 | 1 | 1 | 517ms | 5.68s | lookup_single_dnsbl | Mail::SpamAssassin::Plugin::URIDNSBL::
234 | 1 | 1 | 424ms | 8.36s | query_hosts_or_domains | Mail::SpamAssassin::Plugin::URIDNSBL::
383 | 1 | 1 | 17.6ms | 1.09s | lookup_a_record | Mail::SpamAssassin::Plugin::URIDNSBL::
317 | 1 | 1 | 14.4ms | 892ms | lookup_domain_ns | Mail::SpamAssassin::Plugin::URIDNSBL::
3474 | 9 | 1 | 14.3ms | 14.3ms | CORE:match (opcode) | Mail::SpamAssassin::Plugin::URIDNSBL::
63 | 1 | 1 | 3.70ms | 3.87ms | __ANON__[:783] | Mail::SpamAssassin::Plugin::URIDNSBL::
21 | 1 | 1 | 1.57ms | 3.76ms | __ANON__[:662] | Mail::SpamAssassin::Plugin::URIDNSBL::
22 | 2 | 1 | 1.50ms | 2.14ms | parse_and_canonicalize_subtest | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 142µs | 463µs | set_config | Mail::SpamAssassin::Plugin::URIDNSBL::
23 | 3 | 1 | 118µs | 118µs | CORE:subst (opcode) | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 108µs | 310µs | __ANON__[:609] | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 91µs | 617µs | new | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 78µs | 90µs | __ANON__[:581] | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 47µs | 47µs | BEGIN@295 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 31µs | 215µs | BEGIN@307 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 30µs | 36µs | BEGIN@301 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 28µs | 544µs | BEGIN@296 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 25µs | 87µs | BEGIN@297 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 25µs | 162µs | BEGIN@298 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 24µs | 37µs | BEGIN@299 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 22µs | 59µs | BEGIN@300 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 22µs | 92µs | BEGIN@302 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 22µs | 93µs | BEGIN@304 | Mail::SpamAssassin::Plugin::URIDNSBL::
1 | 1 | 1 | 15µs | 15µs | has_tflags_domains_only | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:1070] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:634] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:687] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:715] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:740] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:768] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:799] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:810] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:913] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | __ANON__[:998] | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | check_uridnsbl | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | complete_a_lookup | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | complete_dnsbl_lookup | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | complete_ns_lookup | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | got_dnsbl_hit | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | has_subtest_for_ranges | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | has_uridnsbl_for_a | Mail::SpamAssassin::Plugin::URIDNSBL::
0 | 0 | 0 | 0s | 0s | lookup_dnsbl_for_ip | Mail::SpamAssassin::Plugin::URIDNSBL::
Line | State ments |
Time on line |
Calls | Time in subs |
Code |
---|---|---|---|---|---|
1 | # <@LICENSE> | ||||
2 | # Licensed to the Apache Software Foundation (ASF) under one or more | ||||
3 | # contributor license agreements. See the NOTICE file distributed with | ||||
4 | # this work for additional information regarding copyright ownership. | ||||
5 | # The ASF licenses this file to you under the Apache License, Version 2.0 | ||||
6 | # (the "License"); you may not use this file except in compliance with | ||||
7 | # the License. You may obtain a copy of the License at: | ||||
8 | # | ||||
9 | # http://www.apache.org/licenses/LICENSE-2.0 | ||||
10 | # | ||||
11 | # Unless required by applicable law or agreed to in writing, software | ||||
12 | # distributed under the License is distributed on an "AS IS" BASIS, | ||||
13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||||
14 | # See the License for the specific language governing permissions and | ||||
15 | # limitations under the License. | ||||
16 | # </@LICENSE> | ||||
17 | |||||
18 | =head1 NAME | ||||
19 | |||||
20 | URIDNSBL - look up URLs against DNS blocklists | ||||
21 | |||||
22 | =head1 SYNOPSIS | ||||
23 | |||||
24 | loadplugin Mail::SpamAssassin::Plugin::URIDNSBL | ||||
25 | uridnsbl URIBL_SBLXBL sbl-xbl.spamhaus.org. TXT | ||||
26 | |||||
27 | =head1 DESCRIPTION | ||||
28 | |||||
29 | This works by analysing message text and HTML for URLs, extracting host | ||||
30 | names from those, then querying various DNS blocklists for either: | ||||
31 | IP addresses of these hosts (uridnsbl,a) or their nameservers (uridnsbl,ns), | ||||
32 | or domain names of these hosts (urirhsbl), or domain names of their | ||||
33 | nameservers (urinsrhsbl, urifullnsrhsbl). | ||||
34 | |||||
35 | =head1 USER SETTINGS | ||||
36 | |||||
37 | =over 4 | ||||
38 | |||||
39 | =item skip_uribl_checks ( 0 | 1 ) (default: 0) | ||||
40 | |||||
41 | Turning on the skip_uribl_checks setting will disable the URIDNSBL plugin. | ||||
42 | |||||
43 | By default, SpamAssassin will run URI DNSBL checks. Individual URI blocklists | ||||
44 | may be disabled selectively by setting a score of a corresponding rule to 0 | ||||
45 | or through the uridnsbl_skip_domain parameter. | ||||
46 | |||||
47 | See also a related configuration parameter skip_rbl_checks, | ||||
48 | which controls the DNSEval plugin (documented in the Conf man page). | ||||
49 | |||||
50 | =back | ||||
51 | |||||
52 | =over 4 | ||||
53 | |||||
54 | =item uridnsbl_skip_domain domain1 domain2 ... | ||||
55 | |||||
56 | Specify a domain, or a number of domains, which should be skipped for the | ||||
57 | URIBL checks. This is very useful to specify very common domains which are | ||||
58 | not going to be listed in URIBLs. | ||||
59 | |||||
60 | =back | ||||
61 | |||||
62 | =over 4 | ||||
63 | |||||
64 | =item clear_uridnsbl_skip_domain [domain1 domain2 ...] | ||||
65 | |||||
66 | If no argument is given, then clears the entire list of domains declared | ||||
67 | by I<uridnsbl_skip_domain> configuration directives so far. Any subsequent | ||||
68 | I<uridnsbl_skip_domain> directives will start creating a new list of skip | ||||
69 | domains. | ||||
70 | |||||
71 | When given a list of domains as arguments, only the specified domains | ||||
72 | are removed from the list of skipped domains. | ||||
73 | |||||
74 | =back | ||||
75 | |||||
76 | =head1 RULE DEFINITIONS AND PRIVILEGED SETTINGS | ||||
77 | |||||
78 | =over 4 | ||||
79 | |||||
80 | =item uridnsbl NAME_OF_RULE dnsbl_zone lookuptype | ||||
81 | |||||
82 | Specify a lookup. C<NAME_OF_RULE> is the name of the rule to be | ||||
83 | used, C<dnsbl_zone> is the zone to look up IPs in, and C<lookuptype> | ||||
84 | is the type of lookup (B<TXT> or B<A>). Note that you must also | ||||
85 | define a body-eval rule calling C<check_uridnsbl()> to use this. | ||||
86 | |||||
87 | This works by collecting domain names from URLs and querying DNS | ||||
88 | blocklists with an IP address of host names found in URLs or with | ||||
89 | IP addresses of their name servers, according to tflags as follows. | ||||
90 | |||||
91 | If the corresponding body rule has a tflag 'a', the DNS blocklist will | ||||
92 | be queried with an IP address of a host found in URLs. | ||||
93 | |||||
94 | If the corresponding body rule has a tflag 'ns', DNS will be queried | ||||
95 | for name servers (NS records) of a domain name found in URLs, then | ||||
96 | these name server names will be resolved to their IP addresses, which | ||||
97 | in turn will be sent to DNS blocklist. | ||||
98 | |||||
99 | Tflags directive may specify either 'a' or 'ns' or both flags. In absence | ||||
100 | of any of these two flags, a default is a 'ns', which is compatible with | ||||
101 | pre-3.4 versions of SpamAssassin. | ||||
102 | |||||
103 | The choice of tflags must correspond to the policy and expected use of | ||||
104 | each DNS blocklist and is normally not a local decision. As an example, | ||||
105 | a blocklist expecting queries resulting from an 'a' tflag is a | ||||
106 | "black_a.txt" ( http://www.uribl.com/datasets.shtml ). | ||||
107 | |||||
108 | Example: | ||||
109 | |||||
110 | uridnsbl URIBL_SBLXBL sbl-xbl.spamhaus.org. TXT | ||||
111 | body URIBL_SBLXBL eval:check_uridnsbl('URIBL_SBLXBL') | ||||
112 | describe URIBL_SBLXBL Contains a URL listed in the SBL/XBL blocklist | ||||
113 | tflags URIBL_SBLXBL net ns | ||||
114 | |||||
115 | =item uridnssub NAME_OF_RULE dnsbl_zone lookuptype subtest | ||||
116 | |||||
117 | Specify a DNSBL-style domain lookup with a sub-test. C<NAME_OF_RULE> is the | ||||
118 | name of the rule to be used, C<dnsbl_zone> is the zone to look up IPs in, | ||||
119 | and C<lookuptype> is the type of lookup (B<TXT> or B<A>). | ||||
120 | |||||
121 | Tflags 'ns' and 'a' on a corresponding body rule are recognized and have | ||||
122 | the same meaning as in the uridnsbl directive. | ||||
123 | |||||
124 | C<subtest> is a sub-test to run against the returned data. The sub-test may | ||||
125 | be in one of the following forms: m, n1-n2, or n/m, where n,n1,n2,m can be | ||||
126 | any of: decimal digits, 0x followed by up to 8 hexadecimal digits, or an IPv4 | ||||
127 | address in quad-dot form. The 'A' records (IPv4 dotted address) as returned | ||||
128 | by DNSBLs lookups are converted into a numerical form (r) and checked against | ||||
129 | the specified sub-test as follows: | ||||
130 | for a range n1-n2 the following must be true: (r >= n1 && r <= n2); | ||||
131 | for a n/m form the following must be true: (r & m) == (n & m); | ||||
132 | for a single value in quad-dot form the following must be true: r == n; | ||||
133 | for a single decimal or hex form the following must be true: | ||||
134 | ((r & n) != 0) && ((r & 0xff000000) == 0x7f000000), i.e. within 127.0.0.0/8 | ||||
135 | |||||
136 | Some typical examples of a sub-test are: 127.0.1.2, 127.0.1.20-127.0.1.39, | ||||
137 | 127.0.1.0/255.255.255.0, 0.0.0.16/0.0.0.16, 0x10/0x10, 16, 0x10 . | ||||
138 | |||||
139 | Note that, as with C<uridnsbl>, you must also define a body-eval rule calling | ||||
140 | C<check_uridnsbl()> to use this. | ||||
141 | |||||
142 | Example: | ||||
143 | |||||
144 | uridnssub URIBL_DNSBL_4 dnsbl.example.org. A 127.0.0.4 | ||||
145 | uridnssub URIBL_DNSBL_8 dnsbl.example.org. A 8 | ||||
146 | |||||
147 | =item urirhsbl NAME_OF_RULE rhsbl_zone lookuptype | ||||
148 | |||||
149 | Specify a RHSBL-style domain lookup. C<NAME_OF_RULE> is the name of the rule | ||||
150 | to be used, C<rhsbl_zone> is the zone to look up domain names in, and | ||||
151 | C<lookuptype> is the type of lookup (B<TXT> or B<A>). Note that you must also | ||||
152 | define a body-eval rule calling C<check_uridnsbl()> to use this. | ||||
153 | |||||
154 | An RHSBL zone is one where the domain name is looked up, as a string; e.g. a | ||||
155 | URI using the domain C<foo.com> will cause a lookup of | ||||
156 | C<foo.com.uriblzone.net>. Note that hostnames are stripped from the domain | ||||
157 | used in the URIBL lookup, so the domain C<foo.bar.com> will look up | ||||
158 | C<bar.com.uriblzone.net>, and C<foo.bar.co.uk> will look up | ||||
159 | C<bar.co.uk.uriblzone.net>. | ||||
160 | |||||
161 | If an URI consists of an IP address instead of a hostname, the IP address is | ||||
162 | looked up (using the standard reversed quads method) in each C<rhsbl_zone>. | ||||
163 | |||||
164 | Example: | ||||
165 | |||||
166 | urirhsbl URIBL_RHSBL rhsbl.example.org. TXT | ||||
167 | |||||
168 | =item urirhssub NAME_OF_RULE rhsbl_zone lookuptype subtest | ||||
169 | |||||
170 | Specify a RHSBL-style domain lookup with a sub-test. C<NAME_OF_RULE> is the | ||||
171 | name of the rule to be used, C<rhsbl_zone> is the zone to look up domain names | ||||
172 | in, and C<lookuptype> is the type of lookup (B<TXT> or B<A>). | ||||
173 | |||||
174 | C<subtest> is a sub-test to run against the returned data. The sub-test may | ||||
175 | be in one of the following forms: m, n1-n2, or n/m, where n,n1,n2,m can be | ||||
176 | any of: decimal digits, 0x followed by up to 8 hexadecimal digits, or an IPv4 | ||||
177 | address in quad-dot form. The 'A' records (IPv4 dotted address) as returned | ||||
178 | by DNSBLs lookups are converted into a numerical form (r) and checked against | ||||
179 | the specified sub-test as follows: | ||||
180 | for a range n1-n2 the following must be true: (r >= n1 && r <= n2); | ||||
181 | for a n/m form the following must be true: (r & m) == (n & m); | ||||
182 | for a single value in quad-dot form the following must be true: r == n; | ||||
183 | for a single decimal or hex form the following must be true: | ||||
184 | ((r & n) != 0) && ((r & 0xff000000) == 0x7f000000), i.e. within 127.0.0.0/8 | ||||
185 | |||||
186 | Some typical examples of a sub-test are: 127.0.1.2, 127.0.1.20-127.0.1.39, | ||||
187 | 127.2.3.0/255.255.255.0, 0.0.0.16/0.0.0.16, 0x10/0x10, 16, 0x10 . | ||||
188 | |||||
189 | Note that, as with C<urirhsbl>, you must also define a body-eval rule calling | ||||
190 | C<check_uridnsbl()> to use this. | ||||
191 | |||||
192 | Example: | ||||
193 | |||||
194 | urirhssub URIBL_RHSBL_4 rhsbl.example.org. A 127.0.0.4 | ||||
195 | urirhssub URIBL_RHSBL_8 rhsbl.example.org. A 8 | ||||
196 | |||||
197 | =item urinsrhsbl NAME_OF_RULE rhsbl_zone lookuptype | ||||
198 | |||||
199 | Perform a RHSBL-style domain lookup against the contents of the NS records | ||||
200 | for each URI. In other words, a URI using the domain C<foo.com> will cause | ||||
201 | an NS lookup to take place; assuming that domain has an NS of C<ns0.bar.com>, | ||||
202 | that will cause a lookup of C<bar.com.uriblzone.net>. Note that hostnames | ||||
203 | are stripped from both the domain used in the URI, and the domain in the | ||||
204 | lookup. | ||||
205 | |||||
206 | C<NAME_OF_RULE> is the name of the rule to be used, C<rhsbl_zone> is the zone | ||||
207 | to look up domain names in, and C<lookuptype> is the type of lookup (B<TXT> or | ||||
208 | B<A>). | ||||
209 | |||||
210 | Note that, as with C<urirhsbl>, you must also define a body-eval rule calling | ||||
211 | C<check_uridnsbl()> to use this. | ||||
212 | |||||
213 | =item urinsrhssub NAME_OF_RULE rhsbl_zone lookuptype subtest | ||||
214 | |||||
215 | Specify a RHSBL-style domain-NS lookup, as above, with a sub-test. | ||||
216 | C<NAME_OF_RULE> is the name of the rule to be used, C<rhsbl_zone> is the zone | ||||
217 | to look up domain names in, and C<lookuptype> is the type of lookup (B<TXT> or | ||||
218 | B<A>). C<subtest> is the sub-test to run against the returned data; see | ||||
219 | <urirhssub>. | ||||
220 | |||||
221 | Note that, as with C<urirhsbl>, you must also define a body-eval rule calling | ||||
222 | C<check_uridnsbl()> to use this. | ||||
223 | |||||
224 | =item urifullnsrhsbl NAME_OF_RULE rhsbl_zone lookuptype | ||||
225 | |||||
226 | Perform a RHSBL-style domain lookup against the contents of the NS records for | ||||
227 | each URI. In other words, a URI using the domain C<foo.com> will cause an NS | ||||
228 | lookup to take place; assuming that domain has an NS of C<ns0.bar.com>, that | ||||
229 | will cause a lookup of C<ns0.bar.com.uriblzone.net>. Note that hostnames are | ||||
230 | stripped from the domain used in the URI. | ||||
231 | |||||
232 | C<NAME_OF_RULE> is the name of the rule to be used, C<rhsbl_zone> is the zone | ||||
233 | to look up domain names in, and C<lookuptype> is the type of lookup (B<TXT> or | ||||
234 | B<A>). | ||||
235 | |||||
236 | Note that, as with C<urirhsbl>, you must also define a body-eval rule calling | ||||
237 | C<check_uridnsbl()> to use this. | ||||
238 | |||||
239 | =item urifullnsrhssub NAME_OF_RULE rhsbl_zone lookuptype subtest | ||||
240 | |||||
241 | Specify a RHSBL-style domain-NS lookup, as above, with a sub-test. | ||||
242 | C<NAME_OF_RULE> is the name of the rule to be used, C<rhsbl_zone> is the zone | ||||
243 | to look up domain names in, and C<lookuptype> is the type of lookup (B<TXT> or | ||||
244 | B<A>). C<subtest> is the sub-test to run against the returned data; see | ||||
245 | <urirhssub>. | ||||
246 | |||||
247 | Note that, as with C<urirhsbl>, you must also define a body-eval rule calling | ||||
248 | C<check_uridnsbl()> to use this. | ||||
249 | |||||
250 | =item tflags NAME_OF_RULE ips_only | ||||
251 | |||||
252 | Only URIs containing IP addresses as the "host" component will be matched | ||||
253 | against the named "urirhsbl"/"urirhssub" rule. | ||||
254 | |||||
255 | =item tflags NAME_OF_RULE domains_only | ||||
256 | |||||
257 | Only URIs containing a non-IP-address "host" component will be matched against | ||||
258 | the named "urirhsbl"/"urirhssub" rule. | ||||
259 | |||||
260 | =item tflags NAME_OF_RULE ns | ||||
261 | |||||
262 | The 'ns' flag may be applied to rules corresponding to uridnsbl and uridnssub | ||||
263 | directives. Host names from URLs will be mapped to their name server IP | ||||
264 | addresses (a NS lookup followed by an A lookup), which in turn will be sent | ||||
265 | to blocklists. This is a default when neither 'a' nor 'ns' flags are specified. | ||||
266 | |||||
267 | =item tflags NAME_OF_RULE a | ||||
268 | |||||
269 | The 'a' flag may be applied to rules corresponding to uridnsbl and uridnssub | ||||
270 | directives. Host names from URLs will be mapped to their IP addresses, which | ||||
271 | will be sent to blocklists. When both 'ns' and 'a' flags are specified, | ||||
272 | both queries will be performed. | ||||
273 | |||||
274 | =back | ||||
275 | |||||
276 | =head1 ADMINISTRATOR SETTINGS | ||||
277 | |||||
278 | =over 4 | ||||
279 | |||||
280 | =item uridnsbl_max_domains N (default: 20) | ||||
281 | |||||
282 | The maximum number of domains to look up. | ||||
283 | |||||
284 | =back | ||||
285 | |||||
286 | =head1 NOTES | ||||
287 | |||||
288 | The C<uridnsbl_timeout> option has been obsoleted by the C<rbl_timeout> | ||||
289 | option. See the C<Mail::SpamAssassin::Conf> POD for details on C<rbl_timeout>. | ||||
290 | |||||
291 | =cut | ||||
292 | |||||
293 | package Mail::SpamAssassin::Plugin::URIDNSBL; | ||||
294 | |||||
295 | 2 | 81µs | 1 | 47µs | # spent 47µs within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@295 which was called:
# once (47µs+0s) by Mail::SpamAssassin::PluginHandler::load_plugin at line 295 # spent 47µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@295 |
296 | 2 | 67µs | 2 | 1.06ms | # spent 544µs (28+517) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@296 which was called:
# once (28µs+517µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 296 # spent 544µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@296
# spent 517µs making 1 call to Exporter::import |
297 | 2 | 60µs | 2 | 148µs | # spent 87µs (25+62) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@297 which was called:
# once (25µs+62µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 297 # spent 87µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@297
# spent 62µs making 1 call to Exporter::import |
298 | 2 | 67µs | 2 | 298µs | # spent 162µs (25+137) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@298 which was called:
# once (25µs+137µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 298 # spent 162µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@298
# spent 137µs making 1 call to Exporter::import |
299 | 2 | 56µs | 2 | 50µs | # spent 37µs (24+13) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@299 which was called:
# once (24µs+13µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 299 # spent 37µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@299
# spent 13µs making 1 call to strict::import |
300 | 2 | 60µs | 2 | 96µs | # spent 59µs (22+37) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@300 which was called:
# once (22µs+37µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 300 # spent 59µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@300
# spent 37µs making 1 call to warnings::import |
301 | 2 | 58µs | 2 | 43µs | # spent 36µs (30+6) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@301 which was called:
# once (30µs+6µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 301 # spent 36µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@301
# spent 6µs making 1 call to bytes::import |
302 | 2 | 66µs | 2 | 162µs | # spent 92µs (22+70) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@302 which was called:
# once (22µs+70µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 302 # spent 92µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@302
# spent 70µs making 1 call to re::import |
303 | |||||
304 | 2 | 94µs | 2 | 164µs | # spent 93µs (22+71) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@304 which was called:
# once (22µs+71µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 304 # spent 93µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@304
# spent 71µs making 1 call to vars::import |
305 | 1 | 29µs | @ISA = qw(Mail::SpamAssassin::Plugin); | ||
306 | |||||
307 | 2 | 9.20ms | 2 | 400µs | # spent 215µs (31+184) within Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@307 which was called:
# once (31µs+184µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 307 # spent 215µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::BEGIN@307
# spent 184µs making 1 call to constant::import |
308 | |||||
309 | # constructor | ||||
310 | # spent 617µs (91+525) within Mail::SpamAssassin::Plugin::URIDNSBL::new which was called:
# once (91µs+525µs) by Mail::SpamAssassin::PluginHandler::load_plugin at line 1 of (eval 35)[Mail/SpamAssassin/PluginHandler.pm:129] | ||||
311 | 1 | 3µs | my $class = shift; | ||
312 | 1 | 3µs | my $samain = shift; | ||
313 | |||||
314 | # some boilerplate... | ||||
315 | 1 | 3µs | $class = ref($class) || $class; | ||
316 | 1 | 15µs | 1 | 25µs | my $self = $class->SUPER::new($samain); # spent 25µs making 1 call to Mail::SpamAssassin::Plugin::new |
317 | 1 | 3µs | bless ($self, $class); | ||
318 | |||||
319 | # this can be effectively global, at least in each process, safely | ||||
320 | |||||
321 | 1 | 10µs | $self->{finished} = { }; | ||
322 | |||||
323 | 1 | 11µs | 1 | 38µs | $self->register_eval_rule ("check_uridnsbl"); # spent 38µs making 1 call to Mail::SpamAssassin::Plugin::register_eval_rule |
324 | 1 | 10µs | 1 | 463µs | $self->set_config($samain->{conf}); # spent 463µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::set_config |
325 | |||||
326 | 1 | 10µs | return $self; | ||
327 | } | ||||
328 | |||||
329 | # this is just a placeholder; in fact the results are dealt with later | ||||
330 | sub check_uridnsbl { | ||||
331 | return 0; | ||||
332 | } | ||||
333 | |||||
334 | # --------------------------------------------------------------------------- | ||||
335 | |||||
336 | # once the metadata is parsed, we can access the URI list. So start off | ||||
337 | # the lookups here! | ||||
338 | # spent 15.0s (602ms+14.4) within Mail::SpamAssassin::Plugin::URIDNSBL::parsed_metadata which was called 234 times, avg 64.1ms/call:
# 234 times (602ms+14.4s) by Mail::SpamAssassin::PluginHandler::callback at line 204 of Mail/SpamAssassin/PluginHandler.pm, avg 64.1ms/call | ||||
339 | 234 | 550µs | my ($self, $opts) = @_; | ||
340 | 234 | 828µs | my $pms = $opts->{permsgstatus}; | ||
341 | 234 | 686µs | my $conf = $pms->{conf}; | ||
342 | |||||
343 | 234 | 781µs | return 0 if $conf->{skip_uribl_checks}; | ||
344 | |||||
345 | 234 | 2.77ms | 234 | 9.63ms | if (!$pms->is_dns_available()) { # spent 9.63ms making 234 calls to Mail::SpamAssassin::PerMsgStatus::is_dns_available, avg 41µs/call |
346 | $self->{dns_not_available} = 1; | ||||
347 | return 0; | ||||
348 | } else { | ||||
349 | # due to re-testing dns may become available after being unavailable | ||||
350 | # DOS: I don't think dns_not_available is even used anymore | ||||
351 | 234 | 738µs | $self->{dns_not_available} = 0; | ||
352 | } | ||||
353 | |||||
354 | 234 | 967µs | $pms->{'uridnsbl_activerules'} = { }; | ||
355 | 234 | 800µs | $pms->{'uridnsbl_hits'} = { }; | ||
356 | 234 | 790µs | $pms->{'uridnsbl_seen_lookups'} = { }; | ||
357 | |||||
358 | # only hit DNSBLs for active rules (defined and score != 0) | ||||
359 | 234 | 827µs | $pms->{'uridnsbl_active_rules_rhsbl'} = { }; | ||
360 | 234 | 843µs | $pms->{'uridnsbl_active_rules_rhsbl_ipsonly'} = { }; | ||
361 | 234 | 764µs | $pms->{'uridnsbl_active_rules_rhsbl_domsonly'} = { }; | ||
362 | 234 | 806µs | $pms->{'uridnsbl_active_rules_nsrhsbl'} = { }; | ||
363 | 234 | 862µs | $pms->{'uridnsbl_active_rules_fullnsrhsbl'} = { }; | ||
364 | 234 | 771µs | $pms->{'uridnsbl_active_rules_nsrevipbl'} = { }; | ||
365 | 234 | 853µs | $pms->{'uridnsbl_active_rules_arevipbl'} = { }; | ||
366 | |||||
367 | 468 | 5.75ms | foreach my $rulename (keys %{$conf->{uridnsbls}}) { | ||
368 | 5382 | 36.9ms | 5382 | 244ms | next unless ($conf->is_rule_active('body_evals',$rulename)); # spent 244ms making 5382 calls to Mail::SpamAssassin::Conf::is_rule_active, avg 45µs/call |
369 | |||||
370 | 5382 | 41.6ms | my $rulecf = $conf->{uridnsbls}->{$rulename}; | ||
371 | 5382 | 33.0ms | my $tflags = $conf->{tflags}->{$rulename}; | ||
372 | 5382 | 8.91ms | $tflags = '' if !defined $tflags; | ||
373 | 13806 | 113ms | my %tfl = map { ($_,1) } split(' ',$tflags); | ||
374 | |||||
375 | 5382 | 28.9ms | my $is_rhsbl = $rulecf->{is_rhsbl}; | ||
376 | 5382 | 35.7ms | if ( $is_rhsbl && $tfl{'ips_only'}) { | ||
377 | $pms->{uridnsbl_active_rules_rhsbl_ipsonly}->{$rulename} = 1; | ||||
378 | } elsif ($is_rhsbl && $tfl{'domains_only'}) { | ||||
379 | 2340 | 15.4ms | $pms->{uridnsbl_active_rules_rhsbl_domsonly}->{$rulename} = 1; | ||
380 | } elsif ($is_rhsbl) { | ||||
381 | 2574 | 17.0ms | $pms->{uridnsbl_active_rules_rhsbl}->{$rulename} = 1; | ||
382 | } elsif ($rulecf->{is_fullnsrhsbl}) { | ||||
383 | $pms->{uridnsbl_active_rules_fullnsrhsbl}->{$rulename} = 1; | ||||
384 | } elsif ($rulecf->{is_nsrhsbl}) { | ||||
385 | $pms->{uridnsbl_active_rules_nsrhsbl}->{$rulename} = 1; | ||||
386 | } else { # just a plain dnsbl rule (IP based), not a RHS rule (name-based) | ||||
387 | 468 | 1.36ms | if ($tfl{'a'}) { # tflag 'a' explicitly | ||
388 | 234 | 750µs | $pms->{uridnsbl_active_rules_arevipbl}->{$rulename} = 1; | ||
389 | } | ||||
390 | 468 | 2.67ms | if ($tfl{'ns'} || !$tfl{'a'}) { # tflag 'ns' explicitly, or default | ||
391 | 234 | 715µs | $pms->{uridnsbl_active_rules_nsrevipbl}->{$rulename} = 1; | ||
392 | } | ||||
393 | } | ||||
394 | } | ||||
395 | |||||
396 | # get all domains in message | ||||
397 | |||||
398 | # don't keep dereferencing this | ||||
399 | 234 | 745µs | my $skip_domains = $conf->{uridnsbl_skip_domains}; | ||
400 | 234 | 468µs | $skip_domains = {} if !$skip_domains; | ||
401 | |||||
402 | # list of hashes to use in order | ||||
403 | 234 | 465µs | my @uri_ordered; | ||
404 | |||||
405 | # Generate the full list of html-parsed domains. | ||||
406 | 234 | 2.52ms | 234 | 5.76s | my $uris = $pms->get_uri_detail_list(); # spent 5.76s making 234 calls to Mail::SpamAssassin::PerMsgStatus::get_uri_detail_list, avg 24.6ms/call |
407 | |||||
408 | # go from uri => info to uri_ordered | ||||
409 | # 0: a | ||||
410 | # 1: form | ||||
411 | # 2: img | ||||
412 | # 3: !a_empty | ||||
413 | # 4: parsed | ||||
414 | # 5: a_empty | ||||
415 | 3024 | 33.6ms | while (my($uri, $info) = each %{$uris}) { | ||
416 | # we want to skip mailto: uris | ||||
417 | 2556 | 31.6ms | 2556 | 9.30ms | next if ($uri =~ /^mailto:/i); # spent 9.30ms making 2556 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match, avg 4µs/call |
418 | |||||
419 | # no hosts/domains were found via this uri, so skip | ||||
420 | 2446 | 6.69ms | next unless ($info->{hosts}); | ||
421 | |||||
422 | 2429 | 4.22ms | my $entry = 3; | ||
423 | |||||
424 | 2429 | 24.2ms | if ($info->{types}->{a}) { | ||
425 | 1011 | 1.68ms | $entry = 5; | ||
426 | |||||
427 | # determine a vs a_empty | ||||
428 | 2022 | 9.84ms | foreach my $at (@{$info->{anchor_text}}) { | ||
429 | 1012 | 2.29ms | if (length $at) { | ||
430 | 1006 | 1.69ms | $entry = 0; | ||
431 | 1006 | 1.78ms | last; | ||
432 | } | ||||
433 | } | ||||
434 | } | ||||
435 | elsif ($info->{types}->{form}) { | ||||
436 | $entry = 1; | ||||
437 | } | ||||
438 | elsif ($info->{types}->{img}) { | ||||
439 | 712 | 1.19ms | $entry = 2; | ||
440 | } | ||||
441 | 690 | 1.38ms | elsif ($info->{types}->{parsed} && (keys %{$info->{types}} == 1)) { | ||
442 | 690 | 1.20ms | $entry = 4; | ||
443 | } | ||||
444 | |||||
445 | # take the usable domains and add them to the ordered list | ||||
446 | 7298 | 54.1ms | while (my($host,$domain) = each( %{$info->{hosts}} )) { | ||
447 | 2440 | 9.15ms | if ($skip_domains->{$domain}) { | ||
448 | 42 | 499µs | 42 | 332µs | dbg("uridnsbl: domain $domain in skip list, host $host"); # spent 332µs making 42 calls to Mail::SpamAssassin::Logger::dbg, avg 8µs/call |
449 | } else { | ||||
450 | # use hostname as a key, and drag along the stripped domain name part | ||||
451 | 2398 | 5.74ms | $uri_ordered[$entry]->{$host} = $domain; | ||
452 | } | ||||
453 | } | ||||
454 | } | ||||
455 | |||||
456 | # at this point, @uri_ordered is an ordered array of hostname hashes | ||||
457 | |||||
458 | 234 | 545µs | my %hostlist; # keys are host names, values are their domain parts | ||
459 | |||||
460 | 234 | 647µs | my $umd = $conf->{uridnsbl_max_domains}; | ||
461 | 234 | 2.76ms | while (keys %hostlist < $umd && @uri_ordered) { | ||
462 | 720 | 1.37ms | my $array = shift @uri_ordered; | ||
463 | 720 | 2.32ms | next unless $array; | ||
464 | |||||
465 | # run through and find the new domains in this grouping | ||||
466 | 700 | 3.91ms | my @hosts = grep(!$hostlist{$_}, keys %{$array}); | ||
467 | 350 | 987µs | next unless @hosts; | ||
468 | |||||
469 | # the new hosts are all useful, just add them in | ||||
470 | 280 | 2.92ms | if (keys(%hostlist) + @hosts <= $umd) { | ||
471 | 280 | 1.17ms | foreach my $host (@hosts) { | ||
472 | 383 | 1.81ms | $hostlist{$host} = $array->{$host}; | ||
473 | } | ||||
474 | } | ||||
475 | else { | ||||
476 | dbg("uridnsbl: more than $umd URIs, picking a subset"); | ||||
477 | # trim down to a limited number - pick randomly | ||||
478 | while (@hosts && keys %hostlist < $umd) { | ||||
479 | my $r = int rand(scalar @hosts); | ||||
480 | my $picked_host = splice(@hosts, $r, 1); | ||||
481 | $hostlist{$picked_host} = $array->{$picked_host}; | ||||
482 | } | ||||
483 | } | ||||
484 | } | ||||
485 | |||||
486 | 234 | 1.05ms | my @hnames = keys %hostlist; | ||
487 | 234 | 1.83ms | 155 | 10.1ms | $pms->set_tag('URIHOSTS', # spent 10.1ms making 155 calls to Mail::SpamAssassin::PerMsgStatus::set_tag, avg 65µs/call |
488 | @hnames == 1 ? $hnames[0] : \@hnames) if @hnames; | ||||
489 | 234 | 921µs | my @dnames = values %hostlist; | ||
490 | 234 | 1.49ms | 155 | 8.74ms | $pms->set_tag('URIDOMAINS', # spent 8.74ms making 155 calls to Mail::SpamAssassin::PerMsgStatus::set_tag, avg 56µs/call |
491 | @dnames == 1 ? $dnames[0] : \@dnames) if @dnames; | ||||
492 | |||||
493 | # and query | ||||
494 | 234 | 2.75ms | 234 | 8.36s | $self->query_hosts_or_domains($pms, \%hostlist); # spent 8.36s making 234 calls to Mail::SpamAssassin::Plugin::URIDNSBL::query_hosts_or_domains, avg 35.7ms/call |
495 | |||||
496 | 234 | 3.01ms | return 1; | ||
497 | } | ||||
498 | |||||
499 | # Accepts argument in one of the following forms: m, n1-n2, or n/m, | ||||
500 | # where n,n1,n2,m can be any of: decimal digits, 0x followed by up to 8 | ||||
501 | # hexadecimal digits, or an IPv4 address in quad-dot form. The argument | ||||
502 | # is checked for syntax (undef is returned on syntax errors), hex numbers | ||||
503 | # are converted to decimal, and quad-dot is converted to decimal, then | ||||
504 | # reassembled into original string delimited by '-' or '/'. As a special | ||||
505 | # backward compatibility measure, a single quad-dot (with no second number) | ||||
506 | # is converted into n-n, to distinguish it from a traditional mask-only form. | ||||
507 | # | ||||
508 | # In practice, arguments like the following are anticipated: | ||||
509 | # 127.0.1.2 (same as 127.0.1.2-127.0.1.2 or 127.0.1.2/255.255.255.255) | ||||
510 | # 127.0.1.20-127.0.1.39 (= 0x7f000114-0x7f000127 or 2130706708-2130706727) | ||||
511 | # 0.0.0.16/0.0.0.16 (same as 0x10/0x10 or 16/0x10 or 16/16) | ||||
512 | # 16 (traditional style mask-only, same as 0x10) | ||||
513 | # | ||||
514 | # spent 2.14ms (1.50+633µs) within Mail::SpamAssassin::Plugin::URIDNSBL::parse_and_canonicalize_subtest which was called 22 times, avg 97µs/call:
# 21 times (1.40ms+554µs) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:662] at line 649, avg 93µs/call
# once (107µs+79µs) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:609] at line 596 | ||||
515 | 22 | 58µs | my($subtest) = @_; | ||
516 | 22 | 40µs | my $digested_subtest; | ||
517 | |||||
518 | 22 | 78µs | local($1,$2,$3); | ||
519 | 22 | 303µs | 22 | 101µs | if ($subtest =~ m{^ ([^/-]+) (?: ([/-]) (.+) )? \z}xs) { # spent 101µs making 22 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match, avg 5µs/call |
520 | 22 | 104µs | my($n1,$delim,$n2) = ($1,$2,$3); | ||
521 | 22 | 34µs | my $any_quad_dot; | ||
522 | 22 | 100µs | for ($n1,$n2) { | ||
523 | 44 | 631µs | 44 | 167µs | if (!defined $_) { # spent 167µs making 44 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match, avg 4µs/call |
524 | # ok, $n2 may not exist | ||||
525 | } elsif (/^\d{1,10}\z/) { | ||||
526 | # ok, already a decimal number | ||||
527 | } elsif (/^0x[0-9a-zA-Z]{1,8}\z/) { | ||||
528 | $_ = hex($_); # hex -> number | ||||
529 | } elsif (/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\z/) { | ||||
530 | 11 | 82µs | 11 | 365µs | $_ = Mail::SpamAssassin::Util::my_inet_aton($_); # quad-dot -> number # spent 365µs making 11 calls to Mail::SpamAssassin::Util::my_inet_aton, avg 33µs/call |
531 | 11 | 29µs | $any_quad_dot = 1; | ||
532 | } else { | ||||
533 | return; | ||||
534 | } | ||||
535 | } | ||||
536 | 22 | 127µs | $digested_subtest = defined $n2 ? $n1.$delim.$n2 | ||
537 | : $any_quad_dot ? $n1.'-'.$n1 : "$n1"; | ||||
538 | } | ||||
539 | 22 | 221µs | return $digested_subtest; | ||
540 | } | ||||
541 | |||||
542 | # spent 463µs (142+321) within Mail::SpamAssassin::Plugin::URIDNSBL::set_config which was called:
# once (142µs+321µs) by Mail::SpamAssassin::Plugin::URIDNSBL::new at line 324 | ||||
543 | 1 | 2µs | my($self, $conf) = @_; | ||
544 | 1 | 2µs | my @cmds; | ||
545 | |||||
546 | 1 | 7µs | push(@cmds, { | ||
547 | setting => 'skip_uribl_checks', | ||||
548 | default => 0, | ||||
549 | type => $Mail::SpamAssassin::Conf::CONF_TYPE_BOOL, | ||||
550 | }); | ||||
551 | |||||
552 | 1 | 7µs | push(@cmds, { | ||
553 | setting => 'uridnsbl_max_domains', | ||||
554 | is_admin => 1, | ||||
555 | default => 20, | ||||
556 | type => $Mail::SpamAssassin::Conf::CONF_TYPE_NUMERIC, | ||||
557 | }); | ||||
558 | |||||
559 | push (@cmds, { | ||||
560 | setting => 'uridnsbl', | ||||
561 | is_priv => 1, | ||||
562 | # spent 90µs (78+12) within Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:581] which was called:
# once (78µs+12µs) by Mail::SpamAssassin::Conf::Parser::parse at line 438 of Mail/SpamAssassin/Conf/Parser.pm | ||||
563 | 1 | 6µs | my ($self, $key, $value, $line) = @_; | ||
564 | 1 | 4µs | local($1,$2,$3); | ||
565 | 1 | 37µs | 1 | 6µs | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)$/) { # spent 6µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match |
566 | 1 | 4µs | my $rulename = $1; | ||
567 | 1 | 3µs | my $zone = $2; | ||
568 | 1 | 3µs | my $type = $3; | ||
569 | 1 | 23µs | 1 | 6µs | $zone =~ s/\.\z//; # strip a redundant trailing dot # spent 6µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:subst |
570 | 1 | 9µs | $self->{uridnsbls}->{$rulename} = { | ||
571 | zone => $zone, type => $type, | ||||
572 | is_rhsbl => 0 | ||||
573 | }; | ||||
574 | } | ||||
575 | elsif ($value =~ /^$/) { | ||||
576 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
577 | } | ||||
578 | else { | ||||
579 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
580 | } | ||||
581 | } | ||||
582 | 1 | 16µs | }); | ||
583 | |||||
584 | push (@cmds, { | ||||
585 | setting => 'uridnssub', | ||||
586 | is_priv => 1, | ||||
587 | # spent 310µs (108+201) within Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:609] which was called:
# once (108µs+201µs) by Mail::SpamAssassin::Conf::Parser::parse at line 438 of Mail/SpamAssassin/Conf/Parser.pm | ||||
588 | 1 | 7µs | my ($self, $key, $value, $line) = @_; | ||
589 | 1 | 7µs | local($1,$2,$3,$4); | ||
590 | 1 | 52µs | 1 | 9µs | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)\s+(.*?)\s*$/) { # spent 9µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match |
591 | 1 | 4µs | my $rulename = $1; | ||
592 | 1 | 3µs | my $zone = $2; | ||
593 | 1 | 3µs | my $type = $3; | ||
594 | 1 | 3µs | my $subrule = $4; | ||
595 | 1 | 19µs | 1 | 6µs | $zone =~ s/\.\z//; # strip a redundant trailing dot # spent 6µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:subst |
596 | 1 | 10µs | 1 | 186µs | $subrule = parse_and_canonicalize_subtest($subrule); # spent 186µs making 1 call to Mail::SpamAssassin::Plugin::URIDNSBL::parse_and_canonicalize_subtest |
597 | 1 | 2µs | defined $subrule or return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||
598 | 1 | 14µs | $self->{uridnsbls}->{$rulename} = { | ||
599 | zone => $zone, type => $type, | ||||
600 | is_rhsbl => 0, subtest => $subrule, | ||||
601 | }; | ||||
602 | } | ||||
603 | elsif ($value =~ /^$/) { | ||||
604 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
605 | } | ||||
606 | else { | ||||
607 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
608 | } | ||||
609 | } | ||||
610 | 1 | 17µs | }); | ||
611 | |||||
612 | push (@cmds, { | ||||
613 | setting => 'urirhsbl', | ||||
614 | is_priv => 1, | ||||
615 | code => sub { | ||||
616 | my ($self, $key, $value, $line) = @_; | ||||
617 | local($1,$2,$3); | ||||
618 | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)$/) { | ||||
619 | my $rulename = $1; | ||||
620 | my $zone = $2; | ||||
621 | my $type = $3; | ||||
622 | $zone =~ s/\.\z//; # strip a redundant trailing dot | ||||
623 | $self->{uridnsbls}->{$rulename} = { | ||||
624 | zone => $zone, type => $type, | ||||
625 | is_rhsbl => 1 | ||||
626 | }; | ||||
627 | } | ||||
628 | elsif ($value =~ /^$/) { | ||||
629 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
630 | } | ||||
631 | else { | ||||
632 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
633 | } | ||||
634 | } | ||||
635 | 1 | 9µs | }); | ||
636 | |||||
637 | push (@cmds, { | ||||
638 | setting => 'urirhssub', | ||||
639 | is_priv => 1, | ||||
640 | # spent 3.76ms (1.57+2.19) within Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:662] which was called 21 times, avg 179µs/call:
# 21 times (1.57ms+2.19ms) by Mail::SpamAssassin::Conf::Parser::parse at line 438 of Mail/SpamAssassin/Conf/Parser.pm, avg 179µs/call | ||||
641 | 21 | 114µs | my ($self, $key, $value, $line) = @_; | ||
642 | 21 | 105µs | local($1,$2,$3,$4); | ||
643 | 21 | 515µs | 21 | 134µs | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)\s+(.*?)\s*$/) { # spent 134µs making 21 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match, avg 6µs/call |
644 | 21 | 66µs | my $rulename = $1; | ||
645 | 21 | 54µs | my $zone = $2; | ||
646 | 21 | 54µs | my $type = $3; | ||
647 | 21 | 52µs | my $subrule = $4; | ||
648 | 21 | 262µs | 21 | 106µs | $zone =~ s/\.\z//; # strip a redundant trailing dot # spent 106µs making 21 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:subst, avg 5µs/call |
649 | 21 | 158µs | 21 | 1.95ms | $subrule = parse_and_canonicalize_subtest($subrule); # spent 1.95ms making 21 calls to Mail::SpamAssassin::Plugin::URIDNSBL::parse_and_canonicalize_subtest, avg 93µs/call |
650 | 21 | 39µs | defined $subrule or return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||
651 | 21 | 361µs | $self->{uridnsbls}->{$rulename} = { | ||
652 | zone => $zone, type => $type, | ||||
653 | is_rhsbl => 1, subtest => $subrule, | ||||
654 | }; | ||||
655 | } | ||||
656 | elsif ($value =~ /^$/) { | ||||
657 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
658 | } | ||||
659 | else { | ||||
660 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
661 | } | ||||
662 | } | ||||
663 | 1 | 10µs | }); | ||
664 | |||||
665 | push (@cmds, { | ||||
666 | setting => 'urinsrhsbl', | ||||
667 | is_priv => 1, | ||||
668 | code => sub { | ||||
669 | my ($self, $key, $value, $line) = @_; | ||||
670 | local($1,$2,$3); | ||||
671 | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)$/) { | ||||
672 | my $rulename = $1; | ||||
673 | my $zone = $2; | ||||
674 | my $type = $3; | ||||
675 | $zone =~ s/\.\z//; # strip a redundant trailing dot | ||||
676 | $self->{uridnsbls}->{$rulename} = { | ||||
677 | zone => $zone, type => $type, | ||||
678 | is_nsrhsbl => 1 | ||||
679 | }; | ||||
680 | } | ||||
681 | elsif ($value =~ /^$/) { | ||||
682 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
683 | } | ||||
684 | else { | ||||
685 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
686 | } | ||||
687 | } | ||||
688 | 1 | 9µs | }); | ||
689 | |||||
690 | push (@cmds, { | ||||
691 | setting => 'urinsrhssub', | ||||
692 | is_priv => 1, | ||||
693 | code => sub { | ||||
694 | my ($self, $key, $value, $line) = @_; | ||||
695 | local($1,$2,$3,$4); | ||||
696 | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)\s+(.*?)\s*$/) { | ||||
697 | my $rulename = $1; | ||||
698 | my $zone = $2; | ||||
699 | my $type = $3; | ||||
700 | my $subrule = $4; | ||||
701 | $zone =~ s/\.\z//; # strip a redundant trailing dot | ||||
702 | $subrule = parse_and_canonicalize_subtest($subrule); | ||||
703 | defined $subrule or return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
704 | $self->{uridnsbls}->{$rulename} = { | ||||
705 | zone => $zone, type => $type, | ||||
706 | is_nsrhsbl => 1, subtest => $subrule, | ||||
707 | }; | ||||
708 | } | ||||
709 | elsif ($value =~ /^$/) { | ||||
710 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
711 | } | ||||
712 | else { | ||||
713 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
714 | } | ||||
715 | } | ||||
716 | 1 | 8µs | }); | ||
717 | |||||
718 | push (@cmds, { | ||||
719 | setting => 'urifullnsrhsbl', | ||||
720 | is_priv => 1, | ||||
721 | code => sub { | ||||
722 | my ($self, $key, $value, $line) = @_; | ||||
723 | local($1,$2,$3); | ||||
724 | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)$/) { | ||||
725 | my $rulename = $1; | ||||
726 | my $zone = $2; | ||||
727 | my $type = $3; | ||||
728 | $zone =~ s/\.\z//; # strip a redundant trailing dot | ||||
729 | $self->{uridnsbls}->{$rulename} = { | ||||
730 | zone => $zone, type => $type, | ||||
731 | is_fullnsrhsbl => 1 | ||||
732 | }; | ||||
733 | } | ||||
734 | elsif ($value =~ /^$/) { | ||||
735 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
736 | } | ||||
737 | else { | ||||
738 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
739 | } | ||||
740 | } | ||||
741 | 1 | 8µs | }); | ||
742 | |||||
743 | push (@cmds, { | ||||
744 | setting => 'urifullnsrhssub', | ||||
745 | is_priv => 1, | ||||
746 | code => sub { | ||||
747 | my ($self, $key, $value, $line) = @_; | ||||
748 | local($1,$2,$3,$4); | ||||
749 | if ($value =~ /^(\S+)\s+(\S+)\s+(\S+)\s+(.*?)\s*$/) { | ||||
750 | my $rulename = $1; | ||||
751 | my $zone = $2; | ||||
752 | my $type = $3; | ||||
753 | my $subrule = $4; | ||||
754 | $zone =~ s/\.\z//; # strip a redundant trailing dot | ||||
755 | $subrule = parse_and_canonicalize_subtest($subrule); | ||||
756 | defined $subrule or return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
757 | $self->{uridnsbls}->{$rulename} = { | ||||
758 | zone => $zone, type => $type, | ||||
759 | is_fullnsrhsbl => 1, subtest => $subrule, | ||||
760 | }; | ||||
761 | } | ||||
762 | elsif ($value =~ /^$/) { | ||||
763 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
764 | } | ||||
765 | else { | ||||
766 | return $Mail::SpamAssassin::Conf::INVALID_VALUE; | ||||
767 | } | ||||
768 | } | ||||
769 | 1 | 8µs | }); | ||
770 | |||||
771 | push (@cmds, { | ||||
772 | setting => 'uridnsbl_skip_domain', | ||||
773 | default => {}, | ||||
774 | type => $Mail::SpamAssassin::Conf::CONF_TYPE_HASH_KEY_VALUE, | ||||
775 | # spent 3.87ms (3.70+168µs) within Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:783] which was called 63 times, avg 61µs/call:
# 63 times (3.70ms+168µs) by Mail::SpamAssassin::Conf::Parser::parse at line 438 of Mail/SpamAssassin/Conf/Parser.pm, avg 61µs/call | ||||
776 | 63 | 286µs | my ($self, $key, $value, $line) = @_; | ||
777 | 63 | 541µs | 63 | 168µs | if ($value =~ /^$/) { # spent 168µs making 63 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match, avg 3µs/call |
778 | return $Mail::SpamAssassin::Conf::MISSING_REQUIRED_VALUE; | ||||
779 | } | ||||
780 | 63 | 827µs | foreach my $domain (split(/\s+/, $value)) { | ||
781 | 215 | 2.31ms | $self->{uridnsbl_skip_domains}->{lc $domain} = 1; | ||
782 | } | ||||
783 | } | ||||
784 | 1 | 8µs | }); | ||
785 | |||||
786 | push (@cmds, { | ||||
787 | setting => 'clear_uridnsbl_skip_domain', | ||||
788 | type => $Mail::SpamAssassin::Conf::CONF_TYPE_HASH_KEY_VALUE, | ||||
789 | code => sub { | ||||
790 | my ($self, $key, $value, $line) = @_; | ||||
791 | if (!defined $value || $value eq '') { | ||||
792 | # clear the entire list | ||||
793 | $self->{uridnsbl_skip_domains} = {}; | ||||
794 | } else { | ||||
795 | foreach my $domain (split(/\s+/, $value)) { | ||||
796 | delete $self->{uridnsbl_skip_domains}->{lc $domain}; | ||||
797 | } | ||||
798 | } | ||||
799 | } | ||||
800 | 1 | 7µs | }); | ||
801 | |||||
802 | # obsolete | ||||
803 | push(@cmds, { | ||||
804 | setting => 'uridnsbl_timeout', | ||||
805 | code => sub { | ||||
806 | # not a lint_warn(), since it's pretty harmless and we don't want | ||||
807 | # to break stuff like sa-update | ||||
808 | warn("config: 'uridnsbl_timeout' is obsolete, use 'rbl_timeout' instead"); | ||||
809 | return 0; | ||||
810 | } | ||||
811 | 1 | 6µs | }); | ||
812 | |||||
813 | 1 | 18µs | 1 | 321µs | $conf->{parser}->register_commands(\@cmds); # spent 321µs making 1 call to Mail::SpamAssassin::Conf::Parser::register_commands |
814 | } | ||||
815 | |||||
816 | # --------------------------------------------------------------------------- | ||||
817 | |||||
818 | # spent 8.36s (424ms+7.94) within Mail::SpamAssassin::Plugin::URIDNSBL::query_hosts_or_domains which was called 234 times, avg 35.7ms/call:
# 234 times (424ms+7.94s) by Mail::SpamAssassin::Plugin::URIDNSBL::parsed_metadata at line 494, avg 35.7ms/call | ||||
819 | 234 | 550µs | my ($self, $pms, $hosthash_ref) = @_; | ||
820 | 234 | 817µs | my $conf = $pms->{conf}; | ||
821 | 234 | 652µs | my $seen_lookups = $pms->{'uridnsbl_seen_lookups'}; | ||
822 | |||||
823 | 234 | 697µs | my $rhsblrules = $pms->{uridnsbl_active_rules_rhsbl}; | ||
824 | 234 | 562µs | my $rhsbliprules = $pms->{uridnsbl_active_rules_rhsbl_ipsonly}; | ||
825 | 234 | 546µs | my $rhsbldomrules = $pms->{uridnsbl_active_rules_rhsbl_domsonly}; | ||
826 | 234 | 580µs | my $nsrhsblrules = $pms->{uridnsbl_active_rules_nsrhsbl}; | ||
827 | 234 | 627µs | my $fullnsrhsblrules = $pms->{uridnsbl_active_rules_fullnsrhsbl}; | ||
828 | 234 | 531µs | my $nsreviprules = $pms->{uridnsbl_active_rules_nsrevipbl}; | ||
829 | 234 | 620µs | my $areviprules = $pms->{uridnsbl_active_rules_arevipbl}; | ||
830 | |||||
831 | 234 | 6.70ms | while (my($host,$domain) = each(%$hosthash_ref)) { | ||
832 | 383 | 1.15ms | $domain = lc $domain; # just in case | ||
833 | 383 | 1.17ms | $host = lc $host; | ||
834 | 383 | 3.79ms | 383 | 2.98ms | dbg("uridnsbl: considering host=$host, domain=$domain"); # spent 2.98ms making 383 calls to Mail::SpamAssassin::Logger::dbg, avg 8µs/call |
835 | 383 | 2.79ms | my $obj = { dom => $domain }; | ||
836 | |||||
837 | 383 | 775µs | my ($is_ip, $single_dnsbl); | ||
838 | 383 | 5.97ms | 383 | 2.42ms | if ($host =~ /^\d+\.\d+\.\d+\.\d+$/) { # spent 2.42ms making 383 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match, avg 6µs/call |
839 | my $IPV4_ADDRESS = IPV4_ADDRESS; | ||||
840 | my $IP_PRIVATE = IP_PRIVATE; | ||||
841 | # only look up the IP if it is public and valid | ||||
842 | if ($host =~ /^$IPV4_ADDRESS$/o && $host !~ /^$IP_PRIVATE$/o) { | ||||
843 | my $obj = { dom => $host }; | ||||
844 | $self->lookup_dnsbl_for_ip($pms, $obj, $host); | ||||
845 | # and check the IP in RHSBLs too | ||||
846 | local($1,$2,$3,$4); | ||||
847 | if ($host =~ /^(\d+)\.(\d+)\.(\d+)\.(\d+)$/) { | ||||
848 | $domain = "$4.$3.$2.$1"; | ||||
849 | $single_dnsbl = 1; | ||||
850 | $is_ip = 1; | ||||
851 | } | ||||
852 | } | ||||
853 | } | ||||
854 | else { | ||||
855 | 383 | 829µs | $single_dnsbl = 1; | ||
856 | } | ||||
857 | |||||
858 | 383 | 2.63ms | if ($single_dnsbl) { | ||
859 | # rule names which look up a domain in the basic RHSBL subset | ||||
860 | 766 | 5.59ms | my @rhsblrules = keys %{$rhsblrules}; | ||
861 | |||||
862 | # and add the "domains_only" and "ips_only" subsets as appropriate | ||||
863 | 383 | 1.64ms | if ($is_ip) { | ||
864 | push @rhsblrules, keys %{$rhsbliprules}; | ||||
865 | } else { | ||||
866 | 766 | 4.05ms | push @rhsblrules, keys %{$rhsbldomrules}; | ||
867 | } | ||||
868 | |||||
869 | 383 | 2.15ms | foreach my $rulename (@rhsblrules) { | ||
870 | 8043 | 32.7ms | my $rulecf = $conf->{uridnsbls}->{$rulename}; | ||
871 | $self->lookup_single_dnsbl($pms, $obj, $rulename, | ||||
872 | 8043 | 83.1ms | 8043 | 5.68s | $domain, $rulecf->{zone}, $rulecf->{type}); # spent 5.68s making 8043 calls to Mail::SpamAssassin::Plugin::URIDNSBL::lookup_single_dnsbl, avg 706µs/call |
873 | |||||
874 | # note that these rules are now underway. important: unless the | ||||
875 | # rule hits, in the current design, these will not be considered | ||||
876 | # "finished" until harvest_dnsbl_queries() completes | ||||
877 | 8043 | 74.4ms | 8043 | 266ms | $pms->register_async_rule_start($rulename); # spent 266ms making 8043 calls to Mail::SpamAssassin::PerMsgStatus::register_async_rule_start, avg 33µs/call |
878 | } | ||||
879 | |||||
880 | # perform NS+A or A queries to look up the domain in the non-RHSBL subset, | ||||
881 | # but only if there are active reverse-IP-URIBL rules | ||||
882 | 383 | 7.77ms | 383 | 2.05ms | if ($host !~ /^\d+\.\d+\.\d+\.\d+$/) { # spent 2.05ms making 383 calls to Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match, avg 5µs/call |
883 | 383 | 5.16ms | if ( !$seen_lookups->{'NS:'.$domain} && | ||
884 | (%$nsreviprules || %$nsrhsblrules || %$fullnsrhsblrules) ) { | ||||
885 | 317 | 1.39ms | $seen_lookups->{'NS:'.$domain} = 1; | ||
886 | 317 | 2.66ms | 317 | 892ms | $self->lookup_domain_ns($pms, $obj, $domain); # spent 892ms making 317 calls to Mail::SpamAssassin::Plugin::URIDNSBL::lookup_domain_ns, avg 2.81ms/call |
887 | } | ||||
888 | 383 | 3.38ms | if (%$areviprules && !$seen_lookups->{'A:'.$host}) { | ||
889 | 383 | 2.31ms | $seen_lookups->{'A:'.$host} = 1; | ||
890 | 383 | 1.63ms | my $obj = { dom => $host }; | ||
891 | 383 | 3.32ms | 383 | 1.09s | $self->lookup_a_record($pms, $obj, $host); # spent 1.09s making 383 calls to Mail::SpamAssassin::Plugin::URIDNSBL::lookup_a_record, avg 2.84ms/call |
892 | 383 | 6.66ms | 383 | 12.5ms | $pms->register_async_rule_start($_) for keys %$areviprules; # spent 12.5ms making 383 calls to Mail::SpamAssassin::PerMsgStatus::register_async_rule_start, avg 33µs/call |
893 | } | ||||
894 | } | ||||
895 | } | ||||
896 | } | ||||
897 | } | ||||
898 | |||||
899 | # --------------------------------------------------------------------------- | ||||
900 | |||||
901 | # spent 892ms (14.4+878) within Mail::SpamAssassin::Plugin::URIDNSBL::lookup_domain_ns which was called 317 times, avg 2.81ms/call:
# 317 times (14.4ms+878ms) by Mail::SpamAssassin::Plugin::URIDNSBL::query_hosts_or_domains at line 886, avg 2.81ms/call | ||||
902 | 317 | 1.00ms | my ($self, $pms, $obj, $dom, $rulename) = @_; | ||
903 | |||||
904 | 317 | 1.56ms | my $key = "NS:" . $dom; | ||
905 | 317 | 2.39ms | my $ent = { | ||
906 | key => $key, zone => $dom, obj => $obj, type => "URI-NS", | ||||
907 | rulename => $rulename, | ||||
908 | }; | ||||
909 | # dig $dom ns | ||||
910 | $ent = $pms->{async}->bgsend_and_start_lookup( | ||||
911 | $dom, 'NS', undef, $ent, | ||||
912 | sub { my ($ent2,$pkt) = @_; | ||||
913 | $self->complete_ns_lookup($pms, $ent2, $pkt, $dom) }, | ||||
914 | 317 | 4.79ms | 317 | 878ms | master_deadline => $pms->{master_deadline} ); # spent 878ms making 317 calls to Mail::SpamAssassin::AsyncLoop::bgsend_and_start_lookup, avg 2.77ms/call |
915 | |||||
916 | 317 | 3.13ms | return $ent; | ||
917 | } | ||||
918 | |||||
919 | sub complete_ns_lookup { | ||||
920 | my ($self, $pms, $ent, $pkt, $dom) = @_; | ||||
921 | |||||
922 | if (!$pkt) { | ||||
923 | # $pkt will be undef if the DNS query was aborted (e.g. timed out) | ||||
924 | dbg("uridnsbl: complete_ns_lookup aborted %s", $ent->{key}); | ||||
925 | return; | ||||
926 | } | ||||
927 | |||||
928 | dbg("uridnsbl: complete_ns_lookup %s", $ent->{key}); | ||||
929 | my $conf = $pms->{conf}; | ||||
930 | my @answer = $pkt->answer; | ||||
931 | |||||
932 | my $IPV4_ADDRESS = IPV4_ADDRESS; | ||||
933 | my $IP_PRIVATE = IP_PRIVATE; | ||||
934 | my $nsrhsblrules = $pms->{uridnsbl_active_rules_nsrhsbl}; | ||||
935 | my $fullnsrhsblrules = $pms->{uridnsbl_active_rules_fullnsrhsbl}; | ||||
936 | my $seen_lookups = $pms->{'uridnsbl_seen_lookups'}; | ||||
937 | |||||
938 | my $j = 0; | ||||
939 | foreach my $rr (@answer) { | ||||
940 | $j++; | ||||
941 | my $str = $rr->string; | ||||
942 | next unless (defined($str) && defined($dom)); | ||||
943 | dbg("uridnsbl: got($j) NS for $dom: $str"); | ||||
944 | |||||
945 | if ($rr->type eq 'NS') { | ||||
946 | my $nsmatch = lc $rr->nsdname; # available since at least Net::DNS 0.14 | ||||
947 | my $nsrhblstr = $nsmatch; | ||||
948 | my $fullnsrhblstr = $nsmatch; | ||||
949 | |||||
950 | if ($nsmatch =~ /^\d+\.\d+\.\d+\.\d+$/) { | ||||
951 | # only look up the IP if it is public and valid | ||||
952 | if ($nsmatch =~ /^$IPV4_ADDRESS$/o && $nsmatch !~ /^$IP_PRIVATE$/o) { | ||||
953 | $self->lookup_dnsbl_for_ip($pms, $ent->{obj}, $nsmatch); | ||||
954 | } | ||||
955 | $nsrhblstr = $nsmatch; | ||||
956 | } | ||||
957 | else { | ||||
958 | if (!$seen_lookups->{'A:'.$nsmatch}) { | ||||
959 | $seen_lookups->{'A:'.$nsmatch} = 1; | ||||
960 | $self->lookup_a_record($pms, $ent->{obj}, $nsmatch); | ||||
961 | } | ||||
962 | $nsrhblstr = $self->{main}->{registryboundaries}->trim_domain($nsmatch); | ||||
963 | } | ||||
964 | |||||
965 | foreach my $rulename (keys %{$nsrhsblrules}) { | ||||
966 | my $rulecf = $conf->{uridnsbls}->{$rulename}; | ||||
967 | $self->lookup_single_dnsbl($pms, $ent->{obj}, $rulename, | ||||
968 | $nsrhblstr, $rulecf->{zone}, $rulecf->{type}); | ||||
969 | |||||
970 | $pms->register_async_rule_start($rulename); | ||||
971 | } | ||||
972 | |||||
973 | foreach my $rulename (keys %{$fullnsrhsblrules}) { | ||||
974 | my $rulecf = $conf->{uridnsbls}->{$rulename}; | ||||
975 | $self->lookup_single_dnsbl($pms, $ent->{obj}, $rulename, | ||||
976 | $fullnsrhblstr, $rulecf->{zone}, $rulecf->{type}); | ||||
977 | |||||
978 | $pms->register_async_rule_start($rulename); | ||||
979 | } | ||||
980 | } | ||||
981 | } | ||||
982 | } | ||||
983 | |||||
984 | # --------------------------------------------------------------------------- | ||||
985 | |||||
986 | # spent 1.09s (17.6ms+1.07) within Mail::SpamAssassin::Plugin::URIDNSBL::lookup_a_record which was called 383 times, avg 2.84ms/call:
# 383 times (17.6ms+1.07s) by Mail::SpamAssassin::Plugin::URIDNSBL::query_hosts_or_domains at line 891, avg 2.84ms/call | ||||
987 | 383 | 1.06ms | my ($self, $pms, $obj, $hname, $rulename) = @_; | ||
988 | |||||
989 | 383 | 1.68ms | my $key = "A:" . $hname; | ||
990 | 383 | 2.91ms | my $ent = { | ||
991 | key => $key, zone => $hname, obj => $obj, type => "URI-A", | ||||
992 | rulename => $rulename, | ||||
993 | }; | ||||
994 | # dig $hname a | ||||
995 | $ent = $pms->{async}->bgsend_and_start_lookup( | ||||
996 | $hname, 'A', undef, $ent, | ||||
997 | sub { my ($ent2,$pkt) = @_; | ||||
998 | $self->complete_a_lookup($pms, $ent2, $pkt, $hname) }, | ||||
999 | 383 | 6.30ms | 383 | 1.07s | master_deadline => $pms->{master_deadline} ); # spent 1.07s making 383 calls to Mail::SpamAssassin::AsyncLoop::bgsend_and_start_lookup, avg 2.79ms/call |
1000 | |||||
1001 | 383 | 3.59ms | return $ent; | ||
1002 | } | ||||
1003 | |||||
1004 | sub complete_a_lookup { | ||||
1005 | my ($self, $pms, $ent, $pkt, $hname) = @_; | ||||
1006 | |||||
1007 | if (!$pkt) { | ||||
1008 | # $pkt will be undef if the DNS query was aborted (e.g. timed out) | ||||
1009 | dbg("uridnsbl: complete_a_lookup aborted %s", $ent->{key}); | ||||
1010 | return; | ||||
1011 | } | ||||
1012 | |||||
1013 | dbg("uridnsbl: complete_a_lookup %s", $ent->{key}); | ||||
1014 | my @answer = $pkt->answer; | ||||
1015 | my $j = 0; | ||||
1016 | foreach my $rr (@answer) { | ||||
1017 | $j++; | ||||
1018 | my $str = $rr->string; | ||||
1019 | if (!defined $hname) { | ||||
1020 | warn "complete_a_lookup-1: $j, (hname is undef), $str"; | ||||
1021 | } elsif (!defined $str) { | ||||
1022 | warn "complete_a_lookup-2: $j, $hname, (str is undef)"; | ||||
1023 | next; | ||||
1024 | } | ||||
1025 | dbg("uridnsbl: complete_a_lookup got(%d) A for %s: %s", $j,$hname,$str); | ||||
1026 | |||||
1027 | if ($rr->type eq 'A') { | ||||
1028 | my $ip_address = $rr->rdatastr; | ||||
1029 | $self->lookup_dnsbl_for_ip($pms, $ent->{obj}, $ip_address); | ||||
1030 | } | ||||
1031 | } | ||||
1032 | } | ||||
1033 | |||||
1034 | # --------------------------------------------------------------------------- | ||||
1035 | |||||
1036 | sub lookup_dnsbl_for_ip { | ||||
1037 | my ($self, $pms, $obj, $ip) = @_; | ||||
1038 | |||||
1039 | local($1,$2,$3,$4); | ||||
1040 | $ip =~ /^(\d+)\.(\d+)\.(\d+)\.(\d+)$/; | ||||
1041 | my $revip = "$4.$3.$2.$1"; | ||||
1042 | |||||
1043 | my $conf = $pms->{conf}; | ||||
1044 | my $tflags = $conf->{tflags}; | ||||
1045 | my $cfns = $pms->{uridnsbl_active_rules_nsrevipbl}; | ||||
1046 | my $cfa = $pms->{uridnsbl_active_rules_arevipbl}; | ||||
1047 | foreach my $rulename (keys %$cfa, keys %$cfns) { | ||||
1048 | my $rulecf = $conf->{uridnsbls}->{$rulename}; | ||||
1049 | |||||
1050 | # ips_only/domains_only lookups should not act on this kind of BL | ||||
1051 | next if defined $tflags->{$rulename} && | ||||
1052 | $tflags->{$rulename} =~ /\b(?:ips_only|domains_only)\b/; | ||||
1053 | |||||
1054 | $self->lookup_single_dnsbl($pms, $obj, $rulename, | ||||
1055 | $revip, $rulecf->{zone}, $rulecf->{type}); | ||||
1056 | } | ||||
1057 | } | ||||
1058 | |||||
1059 | # spent 5.68s (517ms+5.16) within Mail::SpamAssassin::Plugin::URIDNSBL::lookup_single_dnsbl which was called 8043 times, avg 706µs/call:
# 8043 times (517ms+5.16s) by Mail::SpamAssassin::Plugin::URIDNSBL::query_hosts_or_domains at line 872, avg 706µs/call | ||||
1060 | 8043 | 79.3ms | my ($self, $pms, $obj, $rulename, $lookupstr, $dnsbl, $qtype) = @_; | ||
1061 | |||||
1062 | 8043 | 38.7ms | my $key = "DNSBL:" . $lookupstr . ':' . $dnsbl; | ||
1063 | 8043 | 95.5ms | my $ent = { | ||
1064 | key => $key, zone => $dnsbl, obj => $obj, type => 'URI-DNSBL', | ||||
1065 | rulename => $rulename, | ||||
1066 | }; | ||||
1067 | $ent = $pms->{async}->bgsend_and_start_lookup( | ||||
1068 | $lookupstr.".".$dnsbl, $qtype, undef, $ent, | ||||
1069 | sub { my ($ent2,$pkt) = @_; | ||||
1070 | $self->complete_dnsbl_lookup($pms, $ent2, $pkt) }, | ||||
1071 | 8043 | 153ms | 8043 | 5.16s | master_deadline => $pms->{master_deadline} ); # spent 5.16s making 8043 calls to Mail::SpamAssassin::AsyncLoop::bgsend_and_start_lookup, avg 641µs/call |
1072 | |||||
1073 | 8043 | 142ms | return $ent; | ||
1074 | } | ||||
1075 | |||||
1076 | sub complete_dnsbl_lookup { | ||||
1077 | my ($self, $pms, $ent, $pkt) = @_; | ||||
1078 | |||||
1079 | if (!$pkt) { | ||||
1080 | # $pkt will be undef if the DNS query was aborted (e.g. timed out) | ||||
1081 | dbg("uridnsbl: complete_dnsbl_lookup aborted %s %s", | ||||
1082 | $ent->{rulename}, $ent->{key}); | ||||
1083 | return; | ||||
1084 | } | ||||
1085 | |||||
1086 | dbg("uridnsbl: complete_dnsbl_lookup %s %s", $ent->{rulename}, $ent->{key}); | ||||
1087 | my $conf = $pms->{conf}; | ||||
1088 | |||||
1089 | my $zone = $ent->{zone}; | ||||
1090 | my $dom = $ent->{obj}->{dom}; | ||||
1091 | my $rulename = $ent->{rulename}; | ||||
1092 | my $rulecf = $conf->{uridnsbls}->{$rulename}; | ||||
1093 | |||||
1094 | my @subtests; | ||||
1095 | my @answer = $pkt->answer; | ||||
1096 | foreach my $rr (@answer) | ||||
1097 | { | ||||
1098 | my($rdatastr,$rdatanum); | ||||
1099 | my $rr_type = $rr->type; | ||||
1100 | |||||
1101 | if ($rr_type eq 'A') { | ||||
1102 | $rdatastr = $rr->rdatastr; | ||||
1103 | if ($rdatastr =~ m/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/) { | ||||
1104 | $rdatanum = Mail::SpamAssassin::Util::my_inet_aton($rdatastr); | ||||
1105 | } | ||||
1106 | } elsif ($rr_type eq 'TXT') { | ||||
1107 | # txtdata returns a non- zone-file-format encoded result, unlike rdatastr; | ||||
1108 | # avoid space-separated RDATA <character-string> fields if possible; | ||||
1109 | # txtdata provides a list of strings in list context since Net::DNS 0.69 | ||||
1110 | $rdatastr = join('',$rr->txtdata); | ||||
1111 | } else { | ||||
1112 | next; | ||||
1113 | } | ||||
1114 | |||||
1115 | my $subtest = $rulecf->{subtest}; | ||||
1116 | |||||
1117 | dbg("uridnsbl: %s . %s -> %s, %s%s", | ||||
1118 | $dom, $zone, $rdatastr, $rulename, | ||||
1119 | !defined $subtest ? '' : ', subtest:'.$subtest); | ||||
1120 | |||||
1121 | my $match; | ||||
1122 | if (!defined $subtest) { | ||||
1123 | # this zone is a simple rule, not a set of subrules | ||||
1124 | # skip any A record that isn't on 127/8 | ||||
1125 | if ($rr_type eq 'A' && $rdatastr !~ /^127\./) { | ||||
1126 | warn("uridnsbl: bogus rr for domain=$dom, rule=$rulename, id=" . | ||||
1127 | $pkt->header->id." rr=".$rr->string); | ||||
1128 | next; | ||||
1129 | } | ||||
1130 | $match = 1; | ||||
1131 | } elsif ($subtest eq $rdatastr) { | ||||
1132 | $match = 1; | ||||
1133 | } elsif ($subtest =~ m{^ (\d+) (?: ([/-]) (\d+) )? \z}x) { | ||||
1134 | my($n1,$delim,$n2) = ($1,$2,$3); | ||||
1135 | $match = | ||||
1136 | !defined $n2 ? ($rdatanum & $n1) && # mask only | ||||
1137 | (($rdatanum & 0xff000000) == 0x7f000000) # 127/8 | ||||
1138 | : $delim eq '-' ? $rdatanum >= $n1 && $rdatanum <= $n2 # range | ||||
1139 | : $delim eq '/' ? ($rdatanum & $n2) == ($n1 & $n2) # value/mask | ||||
1140 | : 0; | ||||
1141 | |||||
1142 | dbg("uridnsbl: %s . %s -> %s, %s, %08x %s %s", | ||||
1143 | $dom, $zone, $rdatastr, $rulename, $rdatanum, | ||||
1144 | !defined $n2 ? sprintf('& %08x', $n1) | ||||
1145 | : $n1 == $n2 ? sprintf('== %08x', $n1) | ||||
1146 | : sprintf('%08x%s%08x', $n1,$delim,$n2), | ||||
1147 | $match ? 'match' : 'no'); | ||||
1148 | } | ||||
1149 | $self->got_dnsbl_hit($pms, $ent, $rdatastr, $dom, $rulename) if $match; | ||||
1150 | } | ||||
1151 | } | ||||
1152 | |||||
1153 | sub got_dnsbl_hit { | ||||
1154 | my ($self, $pms, $ent, $str, $dom, $rulename) = @_; | ||||
1155 | |||||
1156 | $str =~ s/\s+/ /gs; # long whitespace => short | ||||
1157 | dbg("uridnsbl: domain \"$dom\" listed ($rulename): $str"); | ||||
1158 | |||||
1159 | if (!defined $pms->{uridnsbl_hits}->{$rulename}) { | ||||
1160 | $pms->{uridnsbl_hits}->{$rulename} = { }; | ||||
1161 | }; | ||||
1162 | $pms->{uridnsbl_hits}->{$rulename}->{$dom} = 1; | ||||
1163 | |||||
1164 | if ( $pms->{uridnsbl_active_rules_nsrevipbl}->{$rulename} | ||||
1165 | || $pms->{uridnsbl_active_rules_arevipbl}->{$rulename} | ||||
1166 | || $pms->{uridnsbl_active_rules_nsrhsbl}->{$rulename} | ||||
1167 | || $pms->{uridnsbl_active_rules_fullnsrhsbl}->{$rulename} | ||||
1168 | || $pms->{uridnsbl_active_rules_rhsbl}->{$rulename} | ||||
1169 | || $pms->{uridnsbl_active_rules_rhsbl_ipsonly}->{$rulename} | ||||
1170 | || $pms->{uridnsbl_active_rules_rhsbl_domsonly}->{$rulename}) | ||||
1171 | { | ||||
1172 | # TODO: this needs to handle multiple domain hits per rule | ||||
1173 | $pms->clear_test_state(); | ||||
1174 | my $uris = join (' ', keys %{$pms->{uridnsbl_hits}->{$rulename}}); | ||||
1175 | $pms->test_log ("URIs: $uris"); | ||||
1176 | $pms->got_hit ($rulename, ""); | ||||
1177 | |||||
1178 | # note that this rule has completed (since it got at least 1 hit) | ||||
1179 | $pms->register_async_rule_finish($rulename); | ||||
1180 | } | ||||
1181 | } | ||||
1182 | |||||
1183 | # --------------------------------------------------------------------------- | ||||
1184 | |||||
1185 | # capability checks for "if can()": | ||||
1186 | # | ||||
1187 | 1 | 10µs | # spent 15µs within Mail::SpamAssassin::Plugin::URIDNSBL::has_tflags_domains_only which was called:
# once (15µs+0s) by Mail::SpamAssassin::Conf::Parser::cond_clause_can_or_has at line 595 of Mail/SpamAssassin/Conf/Parser.pm | ||
1188 | sub has_subtest_for_ranges { 1 } | ||||
1189 | sub has_uridnsbl_for_a { 1 } # uridnsbl rules recognize tflags 'a' and 'ns' | ||||
1190 | |||||
1191 | 1 | 14µs | 1; | ||
# spent 14.3ms within Mail::SpamAssassin::Plugin::URIDNSBL::CORE:match which was called 3474 times, avg 4µs/call:
# 2556 times (9.30ms+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::parsed_metadata at line 417, avg 4µs/call
# 383 times (2.42ms+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::query_hosts_or_domains at line 838, avg 6µs/call
# 383 times (2.05ms+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::query_hosts_or_domains at line 882, avg 5µs/call
# 63 times (168µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:783] at line 777, avg 3µs/call
# 44 times (167µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::parse_and_canonicalize_subtest at line 523, avg 4µs/call
# 22 times (101µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::parse_and_canonicalize_subtest at line 519, avg 5µs/call
# 21 times (134µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:662] at line 643, avg 6µs/call
# once (9µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:609] at line 590
# once (6µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:581] at line 565 | |||||
# spent 118µs within Mail::SpamAssassin::Plugin::URIDNSBL::CORE:subst which was called 23 times, avg 5µs/call:
# 21 times (106µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:662] at line 648, avg 5µs/call
# once (6µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:581] at line 569
# once (6µs+0s) by Mail::SpamAssassin::Plugin::URIDNSBL::__ANON__[/usr/local/lib/perl5/site_perl/Mail/SpamAssassin/Plugin/URIDNSBL.pm:609] at line 595 |