SSL for Authentication Sucks

Wednesday, November 26, 2014 

One of the most horrible mistakes made in the early days of the internet was to use SSL (an “HTTPS” connection) for both securing a connection with encryption and verifying that the server you reach matches the URL you entered.

Encryption is necessary so you can’t be spied on by anyone running wireshark on the same hotspot you’re on, something that happens all the time, every day, to everyone connecting to public wifi, which means just about everyone just about any time they take a wifi device out of the house.  It is pretty certain that you – you yourself – have thwarted cybercrime attempts thanks to SSL, not just once but perhaps dozens of times a day, depending on how often you go to Starbucks.

The second purpose, attempting to guarantee that the website you reached is served by the owner of the domain name as verified by some random company you’ve never heard of is an attempt to thwart so-called “Man in the Middle” (MITM) and DNS poisoning attacks.  While these are also fairly easy (especially the latter), they’re both relatively uncommon and the “fix” doesn’t work anyway.

In practice, the “fix” can be detrimental because it gives a false sense of security to that sliver of the population that knows enough to be aware that the browser bar ever shows a green lock or any other indicator of browser trust and not aware enough to realize that the indicator is a lie. It is beyond idiotic that our browsers make a big show of this charade of identity verification with great colorful warnings of non-compliance whenever detected to order to force everyone to pay off the cert mafia and join in the protection racket of pretending that their sites are verified.

I’ve written before why this is counterproductive, but the basic problems is that browsers ship with a set of “root” certificates1You can review a list of the certificates of trusted Certificate Authorities here. Note that the list includes state-agency certificates from countries with controversial human rights records. that they trust for no good reason at all except that there’s a massive payola racket and if you’re a certificate issuer with a distributed accepted CA certificate you can print money by charging people absurd fees for executing a script on your server which, at zero cost to the operator, “signs” their certificate request (oh please, please great cert authority sign my request) so that browsers will accept it without warning.  It isn’t like they actually have the owner of the site come in to their office, show ID, and verify they are who they say they are.  Nobody does that except CACert; which is a free service and, surprise, their root cert is not included in any shipping browser.

Users then will typically “trust” that the site they’re connecting to is actually the one they expected when they typed in a URL.  Except they didn’t type a URL, they clicked on a link and they really have no idea where there browser is going and will not read the URL in the browser bar anyway and bankomurica.com is just as valid as bankofamerica.com, so the typical user has no clue where the browser thinks it is going and a perfectly legit, valid cert can be presented for a confusing (or not really so much) URL.  Typosquatters and pranksters have exploited this very successfully and have proven beyond any doubt that pretending that a URL is an unambiguous identifier is foolish and so too, therefore, is proving that the connection between the browser and the URL hasn’t been hijacked.

Further, law enforcement in most countries require that service providers ensure that it is possible to surreptitiously intercept communications on the web: that is do the exact thing we’re sold that a “valid” certificate makes “impossible.” In practice they get what are called “lawful intercept” certificates which are a bit like fireman’s key that doesn’t compromise your security because only a fireman would ever, ever have one..  Countries change hands and so do these.  If you think you’re a state-level target and certificate signing has any value, you’re actually putting your life at risk.  This is an immense disservice because there will be some people at risk, under surveillance, who will actually pay attention to the green bar and think it means they are safe.  It does not.  They may die.  Really.

Commercial certs can cost thousands of dollars a year and they provide absolutely zero value to the site visitor except making the browser warnings go away so they can visit the site without dismissing meaningless and annoying warnings.  There is absolutely no additional value to the site operator for a commercial cert over a completely free self-signed cert except to make the browser warnings go away for their visitors.  The only entity that benefits from this is the certificate vendor from the fees they charge site operators and for the browser vendor for whatever fees are associated with including their certificates in the browser installer.  You, the internet user, just lose out because small sites don’t use encryption because they can’t afford certs or the hassle and so your security is compromised to make other people rich.

There are far better tools2The hierarchical security model that browsers currently use, referencing a certificate authority, does work well for top-down organizations like companies or the military (oddly, the US Military’s root certificates aren’t included in browsers).  In such a situation, it makes sense for a central authority to dictate what sources are trusted.  It just does not make sense in an unstructured public environment where the “authority” is unknown and their vouch means nothing.  that use a “Web Of Trust” model that was pioneered by PGP back in the early 1990s that actually does have some meaning and is used by CACert, meaning CACert certificates actually have some meaning when they indicate that the site you’re visiting is the one indicated by the URL, but since CACert doesn’t charge and therefore can’t afford to buy into the cert mafia, their root certs are not included in browsers, so you have to install it yourself.

The result is that a small website operator has four options:

  • Give up on security and expose all the content that moves between their server and their visitors to anyone snooping or logging,
  • Use a self-signed cert3If you’re running your own web services, for example a web-interface to your wifi router or a server or some other device with a web interface, it will probably use a self-signed cert and you’ve probably gotten used to clicking through the warnings, which at least diminishes the blackmail value of the browser warnings as people get used to ignoring them.  Installing certificates in Firefox is pretty easy.  It is a major hassle in Chrome or IE (because Chrome, awesome work Google, great job, uses IE’s certificate store, at least on Windows). Self-signed certs are used everywhere in IT management, almost all web-interfaced equipment uses them.   IBM has a fairly concise description of how to install the certs.  Firefox wins.  to encrypt traffic that will generate all sorts of browser warnings for their visitors in an attempt to extort money from them,
  • Use one of the free SSL certificate services that become increasingly annoying to keep up to date and provide absolutely zero authentication value but will encrypt traffic without generating warnings,
  • Use CACert and ask users to be smart enough to install the CACert root certificate and thus actually encrypt and reasonably securely prove ownership.

And, of course, agitate for rationality: Perspectives and the CACert root should ship with every browser install.

Footnotes   [ + ]

1. You can review a list of the certificates of trusted Certificate Authorities here. Note that the list includes state-agency certificates from countries with controversial human rights records.
2. The hierarchical security model that browsers currently use, referencing a certificate authority, does work well for top-down organizations like companies or the military (oddly, the US Military’s root certificates aren’t included in browsers).  In such a situation, it makes sense for a central authority to dictate what sources are trusted.  It just does not make sense in an unstructured public environment where the “authority” is unknown and their vouch means nothing.
3. If you’re running your own web services, for example a web-interface to your wifi router or a server or some other device with a web interface, it will probably use a self-signed cert and you’ve probably gotten used to clicking through the warnings, which at least diminishes the blackmail value of the browser warnings as people get used to ignoring them.  Installing certificates in Firefox is pretty easy.  It is a major hassle in Chrome or IE (because Chrome, awesome work Google, great job, uses IE’s certificate store, at least on Windows). Self-signed certs are used everywhere in IT management, almost all web-interfaced equipment uses them.   IBM has a fairly concise description of how to install the certs.  Firefox wins.
Posted at 15:50:20 UTC

Category: technology

Awesome SSL client cert fun

Tuesday, May 31, 2011 

Client cert authentication is oddly elusive given the practical value. I found a neat bug:

with
SSLVerifyClient optional
SSLVerifyDepth 3
SSLCADNRequestPath /usr/local/openssl/certs/clientcerts/

I get a request for identification in firefox, no problem. If I choose the right certificate to respond with I get an instant child pid 61501 exit signal Bus error (10). Every click on the “OK” button gets another seg fault. Yay. Magic.

client_cert.jpg

signal_bus_error.jpg
Posted at 00:36:09 UTC

Category: FreeBSDtechnology

OpenSSL 1.0.0_4 Install Issues

Sunday, January 16, 2011 

Updating my server’s install of OpenSSL on FreeBSD 8.1 to 1.0.0_4 (openssl-1.0.0c) I ran into a problem where some of the patch files had been updated but the distinfo file in /usr/ports/security/openssl hasn’t (hadn’t?) been updated to match.

The symptoms are that the update of OpenSSL fails with two errors:

  • tls-extractor.patch line count is 1235, distinfo says 1234,
  • dtls-sctp-20.patch SHA256 hash is wrong,
  • (the tls-extractor.patch hash is wrong too, but the line count hits first).

- SHA256 (openssl-1.0.0c/tls-extractor.patch) = bb1aa486327fd96f9d6b870f0a1ad2c83dd4c06a96284eb64dde3f833ba5e0d0
- SIZE (openssl-1.0.0c/tls-extractor.patch) = 1234
- SHA256 (openssl-1.0.0c/dtls-sctp-20.patch) = 3b451618b64d7dbc917942759c26cbc717be3077e9d73cb3c5bd12a82a132268
+ SHA256 (openssl-1.0.0c/tls-extractor.patch) = b7dfb15b6ab7d62348eaa191fc8ba06565c92ecdd5d08bb5e9eb01a2e7433bb2
+ SIZE (openssl-1.0.0c/tls-extractor.patch) = 1235
+ SHA256 (openssl-1.0.0c/dtls-sctp-20.patch) = f002b13fead7c08270a9cfaf556be49c62be5b46f492ad59db29af4d3e9a4e67

Posted at 19:28:58 UTC

Category: FreeBSDtechnology

Unbreaking FireFox SSL Behavior

Sunday, January 24, 2010 

I used to love firefox, but then somebody decided that users were way too stupid to make it through web browsing without an endless parade of  warnings about SSL certs.  The premise seems to be that:

  • Valid certs are meaningful.
  • Self-Signed or expired certs are indicative of a problem.

Neither is true.

(To a statistical certainty.  Some user somewhere will be validly warned away from a phishing site someday.)

Valid certs mean next to nothing since the users that these warnings are targeted to (and me too) will never ever notice if they’re going to bankofamerica.com (or whatever BofA’s legitimate URL is) or bankomerica.com (assuming bankomerica isn’t a valid bank of america domain).  Thus bankomerica can dupe bankofamerica’s website and get a perfectly valid cert and if users were dumb enough to believe that a lack of warnings indicated validity as the huge scary warnings effectively convey, then they’d be easy prey.

The only valid purpose of SSL is to secure communication between a server and a client so you can check your web mail at a cafe without worrying about being snooped and a self-signed cert does that just as well as one issued by the cert mafia.  Sure, sure the giant cert authorities would love to take your $1,000 a year to give a your user’s some sort of guarantee that you’re really who you say you are, but that doesn’t make any difference at all in practice.

As for DNS hijacking so amazon.com goes to a spoof site where the transaction security is compromised (and in theory the self-signed cert would be a give-away) just mod-rewrite to http then redirect to amazoncheck0utservices.com and get a valid cert for it.

Besides, after users have been forced to dismiss a zillion intra-net “invalid” certs, they’ve learned to completely ignore the warnings and so automatically click through the scary and almost always pointless warnings FireFox generates. Or, like many people, users stop abandon the scary, irritating browser and go back to IE.  Win.  Oh wait… FAIL.

Secure DNSSEC is smart, but forget warning people into oblivion over self-signed certs, the net effect is to make the web less secure because site admins have to choose between absurd fees for certs or turning certs off.  Until FireFox fixes this counterproductive behavior, there are two things that help.  First, browse to about:config and set browser.ssl_override_behavior to “2”.

FIX SSL config in FireFox

I’ve also found the Persepectives Plugin useful to reduce the number of pointless and irritating error warnings Firefox generates when it sees a cert that hasn’t fully paid up the protection racket extortion fees by using a polling mechanism, effectively saying (to a collection of referee sites) “ya’ll think this cert is ok?” and if they say “yeah…” then you get no error.

Perspectives_plugin

There fixes are helpful for those of us sufficiently skilled to use them, but unfortunately they won’t prevent users abandoning the endlessly “WOLF!” crying FireFox for IE.

Posted at 19:18:52 UTC

Category: technology

Courier IMAP 4.2.0 breaks SSL-Authd on 993

Sunday, October 14, 2007 

Updating to Courier IMAP 4.2.0_1 broke authentication with SSL on 993 for roundcube (and perhaps others) but not for Thunderbird. The following worked for me:

Change:

TLS_PROTOCOL=SSL3

to:

TLS_PROTOCOL=SSL23

in /usr/local/etc/courier-imap/imapd-ssl

(from this fine site)

Posted at 13:12:47 UTC

Category: FreeBSDtechnology

Let’s Encrypt….

Sunday, December 10, 2017 

Let’s encrypt, why not?

Wanna know how I did it for FreeBSD/Apache/acme-client, jump below.

Let’s encrypt is a service from the fine people at Mozilla, who, when they’re not trying to prove that Firefox can be a Chrome clone, do some really good stuff. Certificates are what give you the little warm fuzzy feeling of a green lock icon, and when properly configured, avoid giving you that terrifying feeling that something horrible is about to happen if you visit a site with an expired one or a self-signed one.

There are some huge structural problems in the certificate concept that seem to exist only to validate the certificate mafia, that can charge $100s per year for a validated certificate, as if executing the script to issue one was somehow expensive. It is not, you can generate one yourself that provides exactly the same security as one provided by a big company that gets their root certs distributed in a browser, but browsers reject these with scary messages so webmasters have to keep buying them.

Now there’s a theory behind why they’re ripping you off: the premise is that the certificate verifies the site is who it says it is – that if you go to mybank.com, you’re actually visiting your real bank, not being redirected by a man-in-the-middle attack to some fake landing page to harvest your passwords, log into your account, and steal all your cats.  There are a few problems with this:

  • Nobody actually checks a URL so while a certificate sort of adds some weight to the probability that mybank.com is owned by mybank, not some hacker a few tables over ARP poisoning the cafe wifi, it doesn’t do anything if you click on a link to mibank.com.
  • The companies that claim to check IDs and verify owners, do not.  That would cost money. You think they’re gonna actually do that?  No… (CAcert actually does, but they don’t get a root cert because… they do it for free. And don’t have Mozilla’s money and clout.)
  • Stealing a root cert private key can generate significant LOLZ; it happens a lot.
  • Law enforcement the world over has “lawful intercept” certs.  You’re probably on some country’s poop list if you have ever used social media. Their laws permit intercepting your communications.  Some country’s laws somewhere certainly do no matter who you are.
  • But dang, those annoying warnings that do nothing to secure you mean that people who publish a website just for the good of the planet either have to pay up, go through a lot of hassle, or leave their user’s content streams exposed to the world’s prying eyes…

…Until Let’s Encrypt came along.  It is a lovely little set of tools and services that not only issue browser-accepted certs (see the green lock?) but also automate renewal.  They basically check that you have enough control over your website to let a script write a file that that they can read back and verify, and if so, you’re who you say you are: the person with write access to the server powering the website they’re giving the certificate too.  That’s all anyone can really do, and is as secure as any other cert there is for identification of a site: that is except for stolen certs, url typos, law enforcement certs, or malicious code on your computer, if you visit https://blackrosetech.com and you don’t get any warnings, you’re probably reading data coming off my computer and not some hacker pretending to be me.

I got Let’s Encrypt to work, but it took some modifications of the existing guides, and I think the service is a good thing that more people should use, so in the spirit of investing some of my resources into the great shared experiment that is Open Source, here’s my How To:


Upstream Guides:

I found these two guides extremely helpful.

https://www.richardfassett.com/2017/01/16/using-lets-encrypt-with-acme-client-on-a-freebsd-11apache-2-4/

https://brnrd.eu/security/2016-12-30/acme-client.html

Step 1: Installing the certificate generation tool

There are a few different software tools to manage the Let’sEncrypt process.  I elected to use Kristaps Dzonsons acme-client, ported to FreeBSD by Bernard Spil.

I was using OpenSSL on my site.  Bernard and Kristaps have some strong opinions on OpenSSL and heartbleed and a few other problems and therefore require LibreSSL.   If you’re using it already, great.  If not, you’ll have to install it.  It wasn’t too terrible, but I ran into a few issues:

https://wiki.freebsd.org/LibreSSL
Or, easy peasy https://ootput.github.io/2016/07/20/Switching-to-LibreSSL/

# ee /etc/make.conf
set  DEFAULT_VERSIONS+= ssl=libressl
# portmaster -od security/libressl security/openssl
# portmaster -rd security/libressl 
or for a complete refresh
# portmaster -Rafd

Curl will probably fail with LibreSSL (and with the latest, if it has brotli support enabled).  Check the google to see if these fixes are still needed, or just:

# cd /usr/ports/ftp/curl
# make config

disable TLS-SRP  https://forums.freebsd.org/threads/56917/

ftp/curl 7.75.0 has an issue with pied piper brotli, which requires modifying the makefile to build --without-brotli as indicated in comment #2 

(Sunpoet, the curl port maintainer, got back to me with an update: when PR/223966 is integrated in Brotli, he will add an optional Brotli support flag and it should work fine at that point without the Makefile edit.)

Step 2: Actually installing acme-client

The really easy part: you should be able to

# portmaster security/acme-client

and be on your way to configuration heaven.

Step 3: Initial configuration

The defaults for acme-client expect certain directories to exist and the installer doesn’t create them.

# mkdir -pm750 /usr/local/www/.well-known && chown -R www:www /usr/local/www/.well-known
# mkdir -pm750 /usr/local/www/.well-known/acme-challenge && chown -R www:www /usr/local/www/.well-known/acme-challenge

The how-to’s seemed to forget the last one.

And make a modification to your httpd.conf file to permit the Let’s Encrypt servers to have access to these folders:

# ee /usr/local/etc/apache24/httpd.conf

add the following:

# Lets Encrypt challenge directory configured per 
# https://brnrd.eu/security/2016-12-30/acme-client.html
<Directory "/usr/local/www/.well-known/">
        Options None
        AllowOverride None
        Require all granted
        Header add Content-Type text/plain
</Directory>

And, for each VHOST that is going to get a cert:

# ee /usr/local/etc/apache24/extra/httpd-vhosts.conf

add to each non-ssl VHOST definition the following:

Alias /.well-known/ /usr/local/www/.well-known/

such that you end up with something like (yours may be different, especially watch out for BasicAuth or ModRewrite, addressed further down):

<VirtualHost IP.NU.MB.ER:80>
    ServerName domain.com
    ServerAdmin admin@domain.com
    DocumentRoot /usr/local/www/data-dist/domain-root
    ServerAlias *.domain.com www.domain.com
    Alias /.well-known/ /usr/local/www/.well-known/
    ErrorLog /var/log/domain-error_log
    CustomLog /var/log/domain-access_log combined
    ScriptAlias /cgi-prg /www/cgi-prg
</VirtualHost>

Don’t forget!

# apachectl restart

Step 4: First Try

At this point the system should be configured sufficiently to do a trial run with a single domain from the command line. Later on there are some scripts that will automate the process of both converting a large number of VHOSTed domains on a server to Let’s Encrypt and for maintaining them and getting email notifications if anything goes wrong in the, hopefully, fully automatic renewal process.

# acme-client -mvnNC /usr/local/www/.well-known/acme-challenge domain.com www.domain.com

This should create all the directories still needed and populate them, then check with the lets encrypt server and get a certificate and install it in the right place.  Inshalla.

If you get something like

acme-client: transfer buffer: [{ "type": "urn:acme:error:malformed","detail": "Provided agreement URL[https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf]does not match current agreement URL[https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf]","status": 400 }] (267 bytes)

That means the lets encrypt agreement has changed.  You can’t do much but write the port maintainer or wait for an update.  It will get fixed quickly and should only happen once a year.  I don’t think you’ll get it at all unless you’re unlucky enough to try to update when it is changing.  I was.

More likely you’ll get something like

acme-client: transfer buffer: [{ "type": "http-01", "status": "invalid", "error": { "type": "urn:acme:error:unauthorized", "detail": "Invalid response from http://www.domain.com/.well-known/acme-challenge/evReZz6s1uSVZbgEVdKkWElx_NHb3NmbbwGbADUwRtQ: (etc...)

This means there’s a problem accessing the /.well-known/ directory by the server.  There can be a lot of reasons for this:

  • You didn’t restart apache # apachectl restart
  • There was an error in the config file (look at the output of the restart) and therefore apache didn’t actually reaload with your new config.
  • DNS isn’t pointing where you think it is pointing.  Check with nslookup/whois to make sure.  Really.
  • You have the directories protected in some way – like with .htaccess.  (see below)

But if it goes well, you’ll get something like:

acme-client: /usr/local/etc/acme/domain.com/privkey.pem: account key exists (not creating)
acme-client: /usr/local/etc/ssl/acme/private/domain.com/privkey.pem: domain key exists (not creating)
acme-client: https://acme-v01.api.letsencrypt.org/directory: directories
acme-client: acme-v01.api.letsencrypt.org: DNS: 173.223.13.221
acme-client: acme-v01.api.letsencrypt.org: DNS: 2001:418:142b:290::3d5
acme-client: acme-v01.api.letsencrypt.org: DNS: 2001:418:142b:28d::3d5
acme-client: https://acme-v01.api.letsencrypt.org/acme/new-authz: req-auth: domain.com
acme-client: /usr/local/www/.well-known/acme-challenge/_ffVe6jHNHbIG1XKAeoqQmmtryWMGCKsfHIWWkl5lJw: created
acme-client: https://acme-v01.api.letsencrypt.org/acme/challenge/5HKzgB9diS5ecS6WbYJsHeEXsSWZeMhdYFmMfN9voHA/2673529867: challenge
acme-client: https://acme-v01.api.letsencrypt.org/acme/challenge/5HKzgB9diS5ecS6WbYJsHeEXsSWZeMhdYFmMfN9voHA/2673529867: status
acme-client: https://acme-v01.api.letsencrypt.org/acme/new-cert: certificate
acme-client: http://cert.int-x3.letsencrypt.org/: full chain
acme-client: cert.int-x3.letsencrypt.org: DNS: 184.23.159.176
acme-client: cert.int-x3.letsencrypt.org: DNS: 184.23.159.177
acme-client: cert.int-x3.letsencrypt.org: DNS: 2001:5a8:100::b817:9fb0
acme-client: cert.int-x3.letsencrypt.org: DNS: 2001:5a8:100::b817:9fb1
acme-client: /usr/local/etc/ssl/acme/domain.com/chain.pem: created
acme-client: /usr/local/etc/ssl/acme/domain.com/cert.pem: created
acme-client: /usr/local/etc/ssl/acme/domain.com/fullchain.pem: created

Yay, you’ve got certs!  Now update your vhosts file to point to the certs you just created.  You may need to add a 443 container or, if it exists, update it to point to the new certs and restart apache.

# ee /usr/local/etc/apache24/extra/httpd-vhosts.conf

<VirtualHost IP.NU.MB.ER:443>
      ServerName domain.com
      ServerAdmin admin@domain.com
      DocumentRoot /usr/local/www/domainroot
      ServerAlias domain.com sub.domain.com
      SSLCertificateFile /usr/local/etc/ssl/acme/domain.com/cert.pem
      SSLCertificateKeyFile /usr/local/etc/ssl/acme/private/domain.com/privkey.pem
      SSLCertificateChainFile /usr/local/etc/ssl/acme/domain.com/fullchain.pem
      Header set Strict-Transport-Security "max-age=31536000; includeSubDomains"
      ErrorLog /var/log/domain-error_log
      CustomLog /var/log/domain-access_log combined
</VirtualHost>

save and restart, look for any errors (typos on directory paths etc. will be detected and apache won’t restart, but be aware, it won’t quit either).

# apachectl restart
 Performing sanity check on apache24 configuration:
 Syntax OK
 Stopping apache24.
 Waiting for PIDS: 81160.
 Performing sanity check on apache24 configuration:
 Syntax OK
 Starting apache24.

Navigate to https://domain.com/ and check out your new green lock.  Check security and you should find:

W00T!

Acme-Client Options

# man acme-client has all the deets, but we’re using:

  • -m to append the domain name to paths, use this and always use it or never.
  • -v for verbose output so we can see what is going on.
  • -n to check if an account key exists and create if not (no reason to omit)
  • -N to check if a domain key exists and create if not (also no reason to omit)
  • -C to specify the path to the challenge dir.  These guides all assume a centralized challenge dir outside the main serving path, and to which we redirect via an alias directive.
  • -F which forces the recreation of certs even if they haven’t expired (this counts against your 10 per 3 hours limit)
  • -s which redirects the process to the Let’s Encrypt staging server, which has no volume limits but also doesn’t create certs browsers accept.  (Using this is fine, but requires cleanup to switch to the production server, see below)
  • -e which is used to add a SAN to the certificate.  Removing one is a bit more involved (see below).

Automating Registration

Lets say you have a lot of domains, you might want to automate the process.  I modified the renewal script to automate the registration process.  This saved some time, but one quirk is you can only register 10 domains (certificates, including SANs, basically 10 lines of the domains list) per 3 hours (they say-I found it takes more like 12 hours to be allowed to register more).

First create  a file with all the domains you want to register for a Let’s Encrypt certificate in the same format as the renewal script uses (it can be the same file, but I made it different as I was experimenting)

# ee /usr/local/etc/acme/newdomains.txt
domain.com www.domain.com
domain2.com www.domain2.com
domain3.com www.domain3.com 
(save)

# ee /usr/local/etc/acme/acme-client-bulk-add.sh

#!/bin/sh

###
#
# This script was adapted by Richard Fassett from letskencrypt.sh
# by Bernard Spil
# See https://brnrd.eu/security/2016-12-30/acme-client.html
#
# and updated again from richard fassett's script at
# https://www.richardfassett.com/2017/01/16/using-lets-encrypt-with-acme-client-on-a-freebsd-11apache-2-4/#comment-282
#
# this requires a file called /usr/local/etc/acme/newdomains.txt of the format
# domain.tld sub.domain.tld alt.domain.tld
# domain2.tld 
# domaind3.tld sub.domain3.tld 
# etc
#
# This should only be run to bulk-add domains.
###

# Define location of dirs and files
DOMAINSFILE="/usr/local/etc/acme/newdomains.txt"
CHALLENGEDIR="/usr/local/www/.well-known/acme-challenge"

# Loop through the newdomains.txt file with lines like
# example.org www.example.org img.example.org
cat ${DOMAINSFILE} | while read domain subdomains ; do

  # Create the cert directory with the command
  # acme-client -mvnNC /usr/local/www/.well-known/acme-challenge (domain subdomains)
  
  acme-client -mvnN -C "${CHALLENGEDIR}" ${domain} ${subdomains}

done

# chmod +x /usr/local/etc/acme/acme-client-bulk-add.sh

A few fixes/recoveries that might be useful at this point: add SAN, remove SAN, switch from staging to production Let’s Encrypt servers.

Automation can break things, you might find you adjusted a few domains incorrectly or want to add a SAN later.

If you need to redo a domain from scratch, for example if you use the “s” option which created a cert from the staging server that doesn’t have volume limits (maybe you’re testing a lot of domains or trying to debug a particularly tricky .htaccess or DNS condition) – you might create a domain with acme-client -mvnsNC /usr/local/www/.well-known/acme-challenge domain.com www.domain.com and then want to generate the production cert.  You also need to do this to remove a SAN.  If you try without deleting the directories, you’ll get something like unknown SAN entry. (You replace “domain.com” with your domain.)

# setenv DD domain.com
# rm -r /usr/local/etc/ssl/acme/private/$DD && rm -r /usr/local/etc/acme/$DD && rm -r /usr/local/etc/ssl/acme/$DD && acme-client -mvnFNC /usr/local/www/.well-known/acme-challenge $DD www.$DD

If you need to add a new SAN to an existing domain

acme-client -mvneFNC /usr/local/www/.well-known/acme-challenge domain.com www.domain.com newsub.domain.com

it is the -e that “extends” the certificate.

Step 5: Automating Renewal

You might notice that the duration of the certificate is rather short: 3 months.  You really don’t want to be responding to certificate expired errors every 3 months, so let’s automate the renewal process.  For this you can create two files and store them on your server.  One is the renewal script itself and the other is a list of domains to renew.  This assumes you have more than one domain.  If you only have one domain, this is a bit overkill, but it will work, so why not?  You might get more domains in the future.   Everyone does.

First create a file with your list of domains, call it something creative like “domains.txt”  This is really a certificate request list with the “primary” domain and Subject Alternative Names (SANs) each on a single line.  In theory the SANs can be all over the place and Let’s Encrypt allows up to 100 per certificate (quite a lot), so the implication of “domains.txt” naming is a bit inaccurate, but that’s what everyone is using so we won’t be contrary.  You have to make sure that all the subdomains resolve—the Let’s Encrypt servers are going to look them up via DNS and if there aren’t working entries, this will fail with one of the errors above.  Check first.  I have not tested whether, if for example, you own domain.com, domain.org, and domain.net and they all point to the same directory, you can use one cert with different TLDs (or domains) as SANs; you should be able to, but I didn’t try.

# ee /usr/local/etc/acme/domains.txt

domain.com www.domain.com sub.domain.com sub2.domain.com
domain.org www.domain.org
domain2.com www.domain2.com cats.domain2.com kittens.domain2.com

Now that you’ve saved that, the following script is adapted from a few at the references listed above and works on my server.  I made a few adjustments and corrections (there was a name change for acme-client which hasn’t quite propagated through all the HowTos yet).

# ee /usr/local/etc/acme/acme-client-update.sh

#!/bin/sh

###
#
# This script was adapted from letskencrypt.sh by Bernard Spil
# See https://brnrd.eu/security/2016-12-30/acme-client.html
# ... and further modified by David Gessel  
# This script will fail if the directories haven't been set up and
# domains in domain.txt have been successfully verified
#
###

# Define location of dirs and files
DOMAINSFILE="/usr/local/etc/acme/domains.txt"
CHALLENGEDIR="/usr/local/www/.well-known/acme-challenge"

# is changed to 1 if any domains expired and were renewed
CHECKEXPIRATION=0

# Loop through the domains.txt file with lines like
# example.org www.example.org img.example.org
cat ${DOMAINSFILE} | while read domain subdomains ; do

    # acme-client returns RC=2 when certificates 
    # weren't changed; use set +e to capture the return code
    set +e
    # Renew the key and certs if required
    acme-client -mvb -C "${CHALLENGEDIR}" ${domain} ${subdomains}
	RC=$?
	
   # now that we have the return code, set script to exit if 
   # nonzero is returned
   set -e

   # if anything is expired, we'll want to do something 
   # (e.g., restart HTTPS)
   if [ $RC -ne 2 ] ; then
        CHECKEXPIRATION=1
   fi
done

if [ "$CHECKEXPIRATION" -ne "0" ] ; then
        service apache24 restart
fi

# chmod +x /usr/local/etc/acme/acme-client-update.sh

This works quite well and will walk through your domains and renew as needed.

I have 36 domain/certificate lines in my “domains.txt” file and timing this script it takes 2.13 seconds to execute on my server.  There’s no real problem running it every night and if you have a lot of domains, you should remember you can only get 10 certs at a time and they won’t renew for about a week before expiry, a limitation I ran into in the bulk setup process.  You can spread your domain renewals out over the three months by force renewing blocks of them if you have more than about 60 per server.

You probably want to automate the process as a cron job. But before we do, lets address one more little problem: one of the shortcomings of the script process below is that the output messages of the script are output to stdout and only cron’s stderr is emailed to the admin. If your shell environment is wrong or the path to the script is wrong, cron will tell you, but if your domains don’t resolve or the script can’t reach /.well-known/, you will not get any warnings. That’s might be a bummer. So I redirect the output of the client-update.sh script to a log file. It gets overwritten with each execution, so it doesn’t need to be rotated – it is just the output of the last execution. It should be filled with lines including “adding SAN” (which it tells you for each domain) and “certificate valid” which it tells you for each cert that doesn’t need to be renewed. But it might tell you something else, like it barfed trying to reach the /.well-known/ directory because, say, you messed around with .htaccess or forgot to renew your domain and it is being redirected to parking or something. The following script first checks to see if there are any lines in /var/log/lets-encrypt-renew other than the expected, and if so, emails just those lines. You shouldn’t get anything until renewal time or if there’s an error. If you don’t care about renewal notices, you can edit the script to ignore those too.

# ee /usr/local/etc/acme/acme-client-errors.sh

#!/bin/sh

###
# this script scans the log file created by the renewal execution cron job
# then removes any lines containing "adding SAN" or "certificate valid", which
# are normal messages, and mails whatever is left over using the "mail" command
# check full paths (or use relative) but full paths can avoid some errors
# use "# which grep" and "# which mail" on your system to check.

PROBLEM=0

/usr/bin/grep -v "adding SAN" /var/log/lets-encrypt-renew | \
/usr/bin/grep -v "certificate valid" | /usr/bin/cat | \
{ while read status
  do
       PROBLEM=1
  done

  if [ "$PROBLEM" -ne "0" ] ; then
        /usr/bin/grep -v "adding SAN" /var/log/lets-encrypt-renew | \
        /usr/bin/grep -v "certificate valid" | \
        /usr/bin/mail -s "Lets Encrypt Errors" gessel@blackrosetech.com $1
  fi
}

# chmod +x /usr/local/etc/acme/acme-client-errors.sh

My cron configuration is set up as

# crontab -e

#*     *     *   *    *        command to be executed
#-     -     -   -    -
#|     |     |   |    |
#|     |     |   |    +----- day of week (0 - 6) (Sunday=0)
#|     |     |   +------- month (1 - 12)
#|     |     +--------- day of        month (1 - 31)
#|     +----------- hour (0 - 23)
#+------------- min (0 - 59)

MAILTO=gessel
# expanded path
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin
SHELL=/bin/csh
# Let's Encrypt renewal check
*       3       *       *       *        /usr/local/etc/acme/acme-client-update.sh > & /var/log/lets-encrypt-renew
*       4       *       *       *        /usr/local/etc/acme/acme-client-errors.sh

Note that this requires that mail works. On servers that aren’t serving email, I use SSMTP and configured it more or less following this guide https://www.freebsd.org/doc/handbook/outgoing-only.html and https://www.davd.eu/freebsd-send-mails-over-an-external-smtp-server/ and this https://www.debarbora.com/freebsd-10-1-setup-ssmtp-for-outgoing-mail/ especially the tip about using # chpass to change the default Full Name for root from “Charlie &” to something useful like “ServerName Root.”

You can test the mail function by adding a random word (or domain) to your domains.txt file and then executing

# /usr/local/etc/acme/acme-client-update.sh > & /var/log/lets-encrypt-renew
# /usr/local/etc/acme/acme-client-errors.sh

If everything is set up right, you’ll get an email complaining about your random word not being valid.  If you restore the correct domains.txt file and execute the above two commands you should not get an email at all.

# more /var/log/lets-encrypt-renew

should show only lines with “adding SAN” and “certificate valid” in them. If you execute # /usr/local/etc/acme/acme-client-errors.sh you shouldn’t get any message.

.htaccess Problems

If you’re controlling access to a directory or have some non-HTML style process listening, you might run into challenges giving the Let’s Encrypt server access to the /.well-known/ directory.  I found the following formulation worked:

AuthType Basic
     AuthName "Please login."
     AuthUserFile "/xxx/.htpasswd"
     # the directive below also "requires" that the requested URL include /.well-known/
        Require expr %{REQUEST_URI} =~ m#^/.well-known/.*#
        Require valid-user

Basically the script above allows (requires) a “valid-user” (one with an entry in the AuthUserFile and valid matching password) and also requires (allows) a URL that is going to /.well-known/ and subdirectories thereof.  This also works in /usr/local/etc/apache24/httpd.conf and /usr/local/etc/apache24/extra/httpd-vhosts.conf

modRewrite to HTTPS problems

You can also create problems by rewriting to HTTPS.  You might want to do this now that you have certs that will auto-renew and you can provide a secure experience for everyone.   In order to get to the /.well-known/ directory, you have to add an exception to the mod-write rule for traffic to this subdirectory like so:

	RewriteEngine on
           RewriteCond %{REQUEST_URI} !^/\.well\-known/acme\-challenge/
	RewriteCond %{HTTPS} off
            RewriteRule (.*) https://%{SERVER_NAME}/$1 [R=301,L]

Also, if you redirect on a 404, some formulations cause problems. This one does not seem to:

ErrorDocument 404 /index.php


Posted at 06:43:58 UTC

Category: FreeBSDHowToSecuritytechnology

A utility for disabling Windows 10

Sunday, June 19, 2016 

While there may be people who actually like Windows 10, there are also many people who aren’t interested in fully exposing every part of their digital life to for-profit mining as means of offsetting Microsoft’s declining profits in the desktop OS business, and if you’re one of those, fighting Microsoft’s truly viral (and malware) marketing techniques is quite a hassle.  It appears there may be an easier way.

Micrsoft has finally provided an “easy” way for people to turn off windows OS update (e.g. from 7.x or 8.x to 10) from happening automatically and without user intervention (and frequently in outright defiance of clear user intent because profits first!)

The short form for people who are comfortable with some of the internal workings of Windows is:

Search for "edit group policy" and open the editor then follow the selection cascade as:

Local Computer Policy -> Computer Configuration -> Administrative Templates 
-> windows Components -> Windows Update 
->> Turn off the upgrade to the latest version... ->> [x] enabled

The longer instructions are at this link: https://support.microsoft.com/en-us/kb/3080351

Easy, no?

I suggest doing this and then downloading and installing the following program.  It will pretty much do the same thing but it also checks to see if Microsoft has already kindly filled your hard disk with malware without your permission and offers to delete it:

https://www.grc.com/never10.htm

Note that my previous posts about removing specific “updates” are still relevant.  The above should prevent windows 10 from auto-updating but Microsoft has been pushing updates with “telemetry” to Win 7 and Win 8, which are also spyware and are tracking you and reporting your usage patterns back to Microsoft without telling you or asking you.

Welcome to the new economy: you’re the product.

Posted at 13:56:56 UTC

Category: uncategorized

Green Lacewings

Sunday, January 10, 2016 

I noticed that my avocado tree was developing brown spots on the leaves, which were almost certainly the result of Persea mites.

Leaf Symptoms

 

So I looked up some possible cures, and it seemed like introducing a predator would be the best option and the least hassle.  I’d had good luck with introduced ladybugs a few years back, which formed a stable population that survived for many years after introduction.  For this pest, green lacewings are recommended.  I found a nearby insectary that could provide larvae on cards and they shipped them overnight.

Green Lacewing egg cards

 

 

The little guys look cute just waiting to hatch…

Green Lacewing Eggs

 

I hung he cards on the leaves of the tree after incubating them overnight in a warm room, and they should hatch sometime in the next day or two, as long as the ants don’t find them first…

Card in tree

 


Update 8 Sept 2016:

The green lacewings seem to have eaten all the mites.  It has been 9 months and there aren’t any signs of damage to this spring’s leaves.  Yay!

No mite bites

The new leaves that grew seem to be developing without any bites at all.  The old leaves that were too damaged have fallen off, but the surviving older leaves still show the scars of the mites.  Green lacewings seem to have done the trick.

Posted at 14:40:47 UTC

Category: photoPositivereviews

Windows 10 Privacy Annihilator

Tuesday, August 4, 2015 

Why would Microsoft, a company whose revenue comes entirely from sales of Windows and Office, start giving Windows 10 away – not just giving it away, but foisting it on users with unbelievably annoying integrated advertisements in the menu of Win 7/8 that pop up endlessly and are tedious to remove and reinstall themselves constantly?

Have they just gone altruistic?  Decided that while they won’t make software free like speech, they’ll make it free like beer? Or is there something more nefarious going on? Something truly horrible, something that will basically screw over the entire windows-using population and sell them off like chattel to any bidder without consent or knowledge?

Of course, it is the latter.

Microsoft is a for-profit company and while their star has been waning lately and they’ve basically ceded the evil empire mantle to Apple, they desperately want to get into the game of stealing your private information and selling it to whoever is willing to pay.

So that’s what Windows 10 does.  It enables Microsoft to steal all of your information, every email, photo, or document you have on your computer and exfiltrate it silently to Microsoft’s servers, and to make it legal they have reserved the right to give it to whoever they want.  This isn’t just the information you stupidly gifted to Google by being dumb enough to use Gmail or ignorantly gifted to Apple by being idiotic enough to load into the iButt, but the files you think are private, on your computer, the ones you don’t upload.  Microsoft gets those.

Finally, we will access, disclose and preserve personal data, including your content (such as the content of your emails, other private communications or files in private folders), when we have a good faith belief that doing so is necessary.

They’ll “access” your data and “disclose” it (meaning to a third party) whenever they have a good faith belief that doing so is necessary.  No warrant needed.  It is necessary for Microsoft to make a buck, so if a  buck is offered for your data, they’re gonna sell it.

If you install Windows 10, you lose. So don’t. If you need to upgrade your operating system, it is time to switch to something that preserves Free like speech: Linux Mint is probably the best choice.

If you’re forced to run Windows 10 for some reason and can’t upgrade to windows 7, then follow these instructions (and these) and remain vigilant, Microsoft’s new strategy is to steal your data and sell it via any backdoor they can sneak past you. Locking them down is going to be a lot of work and might not be possible so keep an eye out for your selfies showing up on pr0n sites: they pay for pix and once you install Windows 10, Microsoft has every right to sell yours.


 

Update: you can’t stop windows 10 from stealing your private data

That’s not quite true – if you never connect your computer to a network, it is very unlikely that Microsoft will be able to secretly exfiltrate your private data through the Windows 10 trojan.  However, it turns out that while the privacy settings do reduce the amount of data that gets sent back to Microsoft, they continue to steal your data even though you’ve told them not to.

Windows 10 is spyware.  It is not an operating system, it is Trojan malware masquerading as an operating system that’s true purpose is to steal your data so Microsoft can sell it without your consent.  If you install Windows 10, you are installing spyware.

Win 10 has apparently been installed 65 million times.  That’s more than 3x as many users’ most intimate, most private data stolen as by the Ashley Madison attack.  If you value privacy, if the idea that you might be denied a loan or insurance because of secret data stolen from your computer without your consent bothers you, if the idea of having evidence of your potential crimes shared with law enforcement without your knowledge and without a warrant worries you then do not install windows 10.  Ever.

Posted at 11:00:30 UTC

Category: Privacytechnology

The CA System is Intractably Broken

Tuesday, July 21, 2015 

I’m dealing with the hassle of setting up certs for a new site over the last few days. It means using startcom’s certs because they’re pretty good (only one security breach) and they have a decently low-hassle free certificate that won’t trigger BS warnings in browsers marketing fake cert mafia placebo security products to unwitting users. (And the CTO answers email within minutes well past midnight.)

And in the middle of this, news of another breach to the CA system was announced on the heels of Lenovo’s SuperFish SSL crack, this time a class break that resulted in a Chinese company being able to generate the equivalent of a lawful intercept cert and provided it to a private company. Official lawful intercept certificates are a globally used tool to silently crack SSL so official governments can monitor SSL encrypted traffic in compliance with national laws like the US’s CALEA.

But this time, it went to a private company and they were using it to intercept and crack Google traffic, and Google found out. The absurdity is to presume that this is an infrequent event. Such breaches (and a “breach” isn’t a lawful intercept tool, which are in constant and widespread use globally, but such a tool in the “wrong” hands) happen regularly. There’s no data on the ratio of discovered breaches to undiscovered breaches, of course. While it is possible that they are always found, seemingly accidental discoveries suggest far wider misuse than generally acknowledged.

The cert mafia should be abolished. Certificate authorities work for authoritarian environments in which a single entity is trusted by fiat as in a dictatorship or a company. The public should trust public opinion and a tool like Perspectives would end these problems as well as significantly lower the barrier to a fully encrypted web as those of us trying to protect our traffic wouldn’t need to choose between forking over cash to the cert mafia for fake security or making our users jump through scary security messages and complex work-arounds.

Posted at 00:53:59 UTC

Category: FreeBSDPrivacySecuritytechnology