Linux

CoreELEC/Kodi on a Cheap S905x Box

Friday, September 11, 2020 

I got a cheap Android TV box off Amazon a while back and consolidated and it ended up being spare.  One of the problems with these TV boxes is they don’t get OS updates after release and soon video apps stop working because of annoying DRM mods.  I figured I’d try to switch one to Linux and see how it went.  There are some complications in that these are ARM devices (not x86/x64) and getting ADB to connect is slightly complicated due to it not being a USB device.  There are a few variations of the “ELEC” (Embedded Linux Entertainment Center), OpenELEC, LibreELEC, and CoreELEC (at least).  CoreELEC focuses on ARM boxes and makes sense for the cheapest boxes.

What you get is a little linux-based device that boots to Kodi, runs an SMB server, and provides SSH access, basically an open source SmartTV that doesn’t spy on you.  It is well suited to a media library you own, rather than rent, such as your DVD/Blu-Ray collection.  Perhaps physical media will come back into vogue as the rent-to-watch streaming world fractures into a million individual subscriptions.  Once moved to the FOSS world, there are regular updates and, generally, people committed to keeping things working and far better and far longer term support than OEM’s offer for random cheap android devices (or, even quite expensive ones).

A: Download the CoreELEC image:

You need to know the processor and RAM of your device – note that a lot of different brands are the same physical boxes.  My “CoCoq M9C Pro” (not listed) is also sold as a “Bqeel M9C Pro” (listed) and the SOC (S905X) and RAM (1G) match, so the Bqeel image worked fine.  The download helper at the bottom of the https://coreelec.org/ site got me the right file easily.

B: Image a uSD or USB device for your wee box:

I’m running Linux as my desktop so I used the LibreELEC Creator Tool and just selected the file downloaded previously.  That part went easily.  The only issue was that after imaging, Linux didn’t recognize the device and I had to systemctl --user restart gvfs-udisks2-volume-monitor before I could do the next step.

C: One manual file move for bootability:

ELEC systems use a “device tree” – a file with hardware descriptors that are device specific to get (most) of the random cheap peripherals working on these cheap boxes.  You have to find the right one in the /Device Trees folder on your newly formatted uSD/USB device and copy it to the root directory and rename it dtb.img.

D: Awesome, now what?

Once you have your configured formatted bootable uSD/USB you have to get your box to boot off it.  This is a bit harder without power/volume control buttons normally on Android devices.  I have ADB set up to strip bloatware from phones.  You just need one command to get this to work, reboot update, but to get it to your box is a bit of a trick.

Most of these boxes are supplied rooted, so you don’t need to do that, but you do need a command line tool and to enable ADB.

To enable ADB: navigate to the build number (usually Settings > System > About phone > Build number) and click it a bunch of times until it says “you’re a developer!”  Then go to developer options and enable ADB.  You can’t use it yet though because there’s no USB.

Install a Terminal Emulator for Android and enable ADB over TCP

su
setprop service.adb.tcp.port 5555
stop adbd
start adbd

Then check your IP with ifconfig and on your desktop computer run

adb connect dev.ip.add.ress:5555

(dev.ip.add.ress is the IP address of your device – like 192.168.100.46, but yours will be different)

Then execute adb reboot update from the desktop computer and the box should reboot then come up in CoreELEC.

Going back to android is as easy as disconnecting the boot media and rebooting, the android OS is still on the device.

Issue: WIFI support.

Many of these cheap devices use the cheapest WIFI chips they can and often the MFGs won’t share details with FOSS developers, so driver support is weak.  Bummer, but the boxes have wired connections and wired works infinitely better anyway: it isn’t a phone, it’s attached to a TV that’s not moving around, run the damn wire and get a stable, reliable connection.  If that’s not possible check the WIFI chips before buying or get a decent USB-WIFI adapter that is supported.

 

Posted at 16:46:48 GMT-0700

Category: HowToLinuxtechnology

WebP and SVG

Tuesday, September 1, 2020 

Using WebP coded images inside SVG containers works.  I haven’t found any automatic way to do it, but it is easy enough manually and results in very efficiently coded images that work well on the internets.  The manual process is to Base64 encode the WebP image and then open the .svg file in a text editor and replace the

xlink:href="data:image/png;base64, ..."

with

xlink:href="data:image/webp;base64,..."

(“…” means the appropriate data, obviously).


Back in about 2010 Google released the spec for WebP, an image compression format that provides a roughly 2-4x coding efficiency over the venerable JPEG (vintage 1974), derived from the VP8 CODEC they bought from ON2. VP8 is a contemporary of and technical equivalent to H.264 and was developed during a rush of innovation to replace the aging MPEG-II standard that included Theora and Dirac. Both VP8 and H.264 CODECs are encumbered by patents, but Google granted an irrevocable license to all patents, making it “open,” while H.264s patents compel licensing from MPEG-LA.  One would think this would tend to make VP8 (and the WEBM container) a global standard, but Apple refused to give Google the win and there’s still no native support in Apple products.

A small aside on video and still coding techniques.

All modern “lossy” (throwing some data away like .mp3, as opposed to “lossless” meaning the original can be reconstructed exactly, as in .flac) CODECs are founded on either Discrete Cosine Transform (DCT) or Wavelet (DWT) encoding of “blocks” of image data.  There are far more detailed write ups online that explain the process in detail, but the basic concept is to divide an image into small tiles of data then apply a mathematical function that converts that data into a form which sorts the information from least human-perceptible to most human-perceptible and sets some threshold for throwing away the least important data while leaving the bits that are most important to human perception.

Wavelets are promising, but never really took off, as in JPEG2000 and Dirac (which was developed by the BBC).  It is a fairly safe bet that any video or still image you see is DCT coded thanks to Nasir Ahmed, T. Natarajan and K. R. Rao.  The differences between 1993’s MPEG-1 and 2013’s H.265 are largely around how the data that is perceptually important is encoded in each still (intra-frame coding) and some very important innovations in inter-frame coding that aren’t relevant to still images.

It is the application of these clever intra-frame perceptual data compression techniques that is most relevant to the coding efficiency difference between JPEG and WebP.

Back to the good stuff…

Way back in 2010 Google experimented with the VP8 intra-coding techniques to develop WebP, a still image CODEC that had to have two core features:

  • better coding efficiency than JPEG,
  • ability to handle transparency like .png or .tiff.

This could be the one standard image coding technique to rule them all – from icons to gigapixel images, all the necessary features and many times better coding efficiency than the rest.  Who wouldn’t love that?

Apple.

Of course it was Apple.  Can’t let Google have the win.  But, finally, with Safari 14 (June 22, 2020 – a decade late!) iOS users can finally see WebP images and websites don’t need crazy auto-detect 1974 tech substitution tricks.  Good job Apple!

It may not be a coincidence that Apple has just released their own still image format based on the intra-frame coding magic of H.265, .heif and maybe they thought it might be a good idea to suddenly pretend to be an open player rather than a walled-garden-screw-you lest iOS insta-users wall themselves off from the 90% of the world that isn’t willing to pay double to pose with a fashionable icon in their hands.  Not surprisingly, .heic, based on H.265 developments is meaningfully more efficient than WebP based on VP8/H.264 era techniques, but as it took WebP 10 years to become a usable web standard, I wouldn’t count on .heic  having universal support soon.  I expect Google is hard at work on a VP9-based version of WebP right now.  Technology is fast, marketing is slow.  Oh well.  In the mean time….

SVG support in browsers is a much older thing – Apple embraced it early (SVG was not developed by Google so….) and basically everything but IE has full support (IE…  the tool you use to download a real browser).  So if we have SVG and WebP, why not both together?

Oddly I can’t find support for this in any of the tools I use, but as noted at the open, it is pretty easy.  The workflow I use is to:

  • generate a graphic in GIMP or Photoshop or whatever and save as .png or .jpg as appropriate to the image content with little compression (high image quality)
  • Combine that with graphics in Inkscape.
  • If the graphics include type, convert the type to SVG paths to avoid font availability problems or having to download a font file before rendering the text or having it render randomly.
  • Save the file (as .svg, the native format of Inkscape)
  • Convert the image file to WebP with a reasonable tool like Nomacs or Ifranview.
  • Base64 encode the image file, either with base64 # base64 infile.webp > outfile.webp.b64 or with this convenient site
  • If you use the command line option the prefix to the data is “data:image/webp;base64,
  • Replace the … on the appropriate xlink:href="...." with the new data using a text editor like Atom.
  • Drop the file on a browser page to see if it works.

WordPress blocks .svg uploads without a plugin, so you need one

The picture is 101.9kB and tolerates zoom quite well. (give it a try, click and zoom on the image).

 

Posted at 08:54:16 GMT-0700

Category: HowToLinuxphotoself-publishingtechnology

Dealing with Apple Branded HEIF .HEIC files on Linux

Saturday, August 22, 2020 

Some of the coding tricks in H.265 have been incorporated into MPEG-H coding, an ISO standard introduced in 2017, which yields a roughly 2:1 coding efficiency gain over the venerable JPEG, which was introduced in 1992.  Remember that?  I do; I’m old.  I remember having a hardware NUBUS JPEG decoder card.   One of the reasons JPEG has lasted so long is that images have become a small storage burden (compared to 4k video, say) and that changing format standards is extremely annoying to everyone.

Apple has elected to make every rational person’s life difficult and put a little barbed wire around their high-fashion walled garden and do something a little special with their brand of a HEVC (h.265) profile for images.  Now normally seeing iOS user’s insta images of how fashionable they are isn’t really worth the effort, but now and then a useful correspondent joins the cult and forks over a ton of money to show off a logo and starts sending you stuff in their special proprietary format.  Annoying, but fixable.

Assuming you’re using an OS that is neither primarily spyware nor fashion forward, such as Linux Mint, you can install HEIF decode (including Apple Brand HEIC) with a few simple commands:

$ sudo add-apt-repository ppa:jakar/qt-heif
$ sudo apt update
$ sudo apt install qt-heif-image-plugin

Once installed, various image viewers should be able to decode the images.  I rather like nomacs as a fairly tolerable replacement for Irfan Skiljan‘s still awesome irfanview.

Posted at 03:56:36 GMT-0700

Category: HowToLinuxphotoPositivereviewstechnology

Update Waterfox with the new PPA on Mint 19.1

Wednesday, June 19, 2019 

The Waterfox PPA changed recently. The following let me update from 56.2.8 to 56.2.10 (between which the old PPA was removed).

First remove the old hawkeye PPA from your sources list, then:

echo "deb http://download.opensuse.org/repositories/home:/hawkeye116477:/waterfox/xUbuntu_18.04/ /" | sudo tee -a /etc/apt/sources.list
wget -nv https://download.opensuse.org/repositories/home:hawkeye116477:waterfox/xUbuntu_18.04/Release.key -O Release.key
sudo apt-key add - < Release.key
sudo apt update
sudo apt upgrade

Note Ubuntu 18.04 = Mint 19/19.1 the 18.10 deb fails.

Read more…

Posted at 11:07:01 GMT-0700

Category: Linux

10 Gbyte Win10 Spyware “upgrade” now forced on users

Sunday, September 27, 2015 

Microsoft has, historically, done some amazingly boneheaded things like clippy, Vista, Win 8, and Win 10.  They have one really good product: Excel, otherwise everything they’ve done has succeeded only through illegal exploitation of an aggressively defended monopoly. OK, maybe the Xbox is competitive, but I’m not much of a gamer.

Sadly for the world, the model of selling users for profit to advertisers and spies has gained ground to the point where Microsoft was starting to look like the least evil major entity in closed-source computing.  Poor microsoft.  To lose the evil crown must be at least as humiliating as their waning revenue and abject failures in the mobile space (so strange… try to enter a space where they don’t have a monopoly to force users to accept their mediocre crap and they fail, who’da thunk it?)

“There is a difference between policy and practice. We don’t read customers mail. We don’t read customer documents. We don’t triangulate YouTube views and searches. We don’t use the content of your Hotmail to target ads in Bing,”

Frank Shaw, Corporate Vice President of Corporate Communications for Microsoft

Well, never fear: Windows 10 is here and they’re radically one-upping the data theft economy by p0wning not just the data you idiotically entrust to someone else’s server for free without ever considering why they’re giving you that useful service for “free” or what they, or whoever buys their ultimately failed business, might do with your data, but also the data you consider too sensitive for the Google or the Apple.  Windows 10 exfiltrates all your data to Microsoft for their use and profit without your information.  Don’t believe it? Read their Privacy Statement.

Finally, we will access, disclose and preserve personal data, including your content (such as the content of your emails, other private communications or files in private folders), when we have a good faith belief that doing so is necessary.

And it is free (as in beer but not as in speech).  What could possiblay go wrong?

Well, people weren’t updating fast enough so Microsoft is now pushing that update on you involuntarily.  Do you have a data cap that a 10G download might break and cost you money?  So what!  Your loss!  Don’t have enough space on your drive for a 10G hidden folder of crapware foisted off on you without your permission?  Tough crap, Microsoft don’t care.

To be clear, Windows 10 is spyware.  If this was coming from a teenage hacker somewhere, they’d be facing jail time.  It is absolutely, unequivocally malware that will create a liability for you if you use it.  If you have any confidentiality requirement, you must not install windows 10.  Ever. Not even on your home machine.  Just don’t.

The only way to prevent this is really annoying and a little risky: disable automatic downloads.  One of the problems with Microsoft’s operating systems is the unbelievably crappy spaghetti code that results in a constant flow of cracks, a week’s worth are patched every Tuesday.  About 1 serious vulnerability every fortnight these days (note this is about the same as Ubuntu and about 1/4 the rate of OSX or iOS, why people think Apple products are “secure” is beyond me – live in that fantasy walled garden!  But nice logo you paid a 50% premium for on your shiny device). Not patching increases the risk that some hacker somewhere will steal your datas, but patching guarantees that Microsoft will steal your datas.  Keep your anti-virus up to date and live a little dangerously by keeping Microsoft out.

Here’s an interesting article: how-to-clean-the-windows-10-crapware-off-your-windows-7-or-81-pc

And a tool referenced in that article: GWX control panel (that can help remove the windows 10 infection if you got it).

And a list of patches I found that are related to Win10 malware that you can remove if you haven’t installed it yet (Windows 10 eliminates the ability to choose or selectively remove patches, once you’re in for the ride, you’re chained in: all or nothing.)

Basic advice:

  • Disable automatic updates and automatic downloads of updates.
  • Review each update Microsoft offers.  This is tedious, my win 7 install reports 384 updates, 5-10 a week, but other than security patches, you probably don’t really need them.  Only install a patch if there’s a reason.  Sorry, that sucks, but there’s always Linux Mint: free like beer AND free like speech.
  • If you’re still on Win 7/8, uninstall the spyware Microsoft has probably already installed.  If you’re on Windows 8, you probably want to upgrade to Windows 7 if at all possible.
  • If you succumbed to the pressure and became a Microsoft Product by installing Windows 10, uninstall it.
  • If uninstall doesn’t work, switch to Mint or reinstall 7.

Most importantly, if you develop software for servers or for end users, stop developing for Microsoft (and Apple too).  Respect the privacy of your customers by not exposing them to exploitation by desperate operating system vendors.  In many classes of applications, your customers buy their computers to run your software: they don’t care what operating system it requires – that should be transparent and painless.  Microsoft is no longer an even remotely acceptable choice.  Server applications should run under FreeBSD or OpenBSD and desktop applications should run under Linux.  You can charge more and generate more profit because the total net cost for your customers will be lower.  Split the difference and give them a more reliable, more secure, and lower cost environment and make more money doing so.

Posted at 08:07:54 GMT-0700

Category: FreeBSDHowToLinuxSecuritytechnology

Successful connect to WPA2 with Linux Mint 17

Saturday, September 26, 2015 

I found myself having odd problems connecting to WPA2 encrypted wireless networks with a new laptop.  There must be more elegant solutions to this problem, but this worked for me.  The problem was that I couldn’t connect to a nearby hotspot secured with WPA2 whether I used the default config tool for mint, Wicd Network Manager, or the command line.  Errors were either “bad password” or the more detailed errors below.

As with any system variation mileage may vary, my errors look like:

wlan0: CTRL-EVENT-SCAN-STARTED 
wlan0: SME: Trying to authenticate with 68:72:51:00:26:26 (SSID='WA-bullet' freq=2462 MHz)
wlan0: Trying to associate with 68:72:51:00:26:26 (SSID='WA-bullet' freq=2462 MHz)
wlan0: Associated with 68:72:51:00:26:26
wlan0: CTRL-EVENT-DISCONNECTED bssid=68:72:51:00:26:26 reason=3 locally_generated=1

and my system config is reported as:

# lspci -vv |grep -i wireless
3e:00.0 Network controller: Intel Corporation Wireless 7260 (rev 6b)
 Subsystem: Intel Corporation Dual Band Wireless-AC 7260
# uname -a
Linux dgzb 3.16.0-38-generic #52~14.04.1-Ubuntu SMP Fri May 8 09:43:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I found useful commands for manually setting up a wpa_supplicant.conf file here, and for disabling 802.11n here. The combination was needed to get things working.

The following successfully connects to a WPA2-secured network:

$ sudo su
$ iw dev
 ... Interface [interfacename] (typically wlan0, assumed below)
$ iw wlan0 scan
 ... SSID: [ssid]
 ... RSN: (if present means the network is secured with WPA2)
$ wpa_passphrase [ssid] >> /etc/wpa_supplicant.conf 
...type in the passphrase for network [ssid] and hit enter...
$ sh -c 'modprobe -r iwlwifi && modprobe iwlwifi 11n_disable=1'
$ wpa_supplicant -i wlan0 -c /etc/wpa_supplicant.conf

(should show CTRL-EVENT-CONNECTED)
(open a new terminal leaving the connection open, ending the command disconnects)

$ sudo su
$ dhclient wlan0

(should be connected now)

Posted at 10:16:28 GMT-0700

Category: HowToLinuxtechnology

A Solution for Mosh Scrollback

Wednesday, July 22, 2015 

Mosh is a pretty good tool, almost indispensable when working in places with crappy internet. While it is designed to help with situations like “LTE on the beach,” it actually works very well in places where internet connectivity is genuinely bad: 1500msec RT, latency, 30% packet loss, and frequent drops in connectivity that last seconds to hours, otherwise known as most of the world. On a good day I lose an SSH connection randomly about every 3-6 hours but I’ve only ever lost a Mosh session when my system went down.

It does a lot of things, but two are key for my use: it syncs user input in the background while local echoing what you type so you can finish your command (and correct a typo) without waiting 1500msec for the remote echo to update; and it creates persistent connections that survive drop off of almost any type except killing the terminal application on one end or the other (anything between can die and when it recovers, you catch up). This means compiles finish and you actually get the output warnings…

…well…

…some of them. Because Mosh’s one giant, glaring, painful, almost debilitating weakness is that it doesn’t support scrollback. So compared to tmux or something else that you can reconnect to after your SSH session drops, you really lose screen content, which is a PITA when ls-ing a directory. I mean, it isn’t that much of an efficiency gain to have to type “ls | less” instead of just “ls” every time you want to see a directory.

I found a solution that works for me. I also use Tmux with Mosh because Tmux will survive a dead client and working with Windows client reboots are a fact of life (I know, sad, but there are some tools I still need on windows, hopefully not for much longer).

Tmux has a facility for creating a local log file, which I then “tail -f” using a separate SSH window. If the SSH client disconnects, no loss, I can pick up the log anytime. It is just mirroring everything that the mosh terminal is doing and the scroll bar scroll back works fine. And it is a raw text file, so you can pipe the output through grep to limit what’s displayed to something of interest and review the log asynchronously as, say, a build is progressing.

Although there are some nice advantages to this, when/if Mosh supports scrollback, it’ll be far more convenient having it in the same window, but for now this is the easiest solution I could come up with.

FreeBSD:

# portmaster sysutils/tmux
# portmaster net/mosh
# ee ~/.tmux.conf
-> bind-key H pipe-pane -o "exec cat >>$HOME/'#W-tmux.log'" \; display-message 'Logging enabled to $HOME/#W-tmux.log'
-> set -g history-limit 30000
Start a Mosh session (for example with Mobaxterm on windows)
# tmux
# [CTRL]-b H
start SSH session (Mobaxterm or Putty on windows)
# tail -f csh-tmux.log
("csh" will be the name of the mosh window - so really "(MoshWindowName)-tmux.log"

You can tmux the ssh session too and still have scrollback and then just reconnect into the same tail command, which preserves the whole scrollback. If you’re on a connection like I’m on, your scrollback logfile will drop off a couple of times a day, but you won’t lose your Mosh session, and it’ll be waiting for you when you’re reminded that you need to see those security warnings from the compile that just scrolled off the Mosh screen forever.

Posted at 00:57:12 GMT-0700

Category: FreeBSDHowToLinuxtechnology

What’s Right About PGP

Thursday, August 14, 2014 

Occasionally you find the crankypants commentary about the “problems” with PGP. These commentaries are invariably written by people who fail to recognize the use modality that PGP is meant to address.

PGP is a cryptographic tool that is, genuinely, annoying to use in most current implementations (though I find the APG extension to the K9 mail app on the android as easy or easier to use than the current Enigmail implementation for Thunderbird.)  The purpose of PGP is to encrypt the contents of mail messages sent between correspondents.  Characteristics of these messages are that they have more than ephemeral value (you might need to reference them again in the future) and that the correspondents are not attempting to hide the fact that they correspond.

It is intrinsic to the capabilities of the tool that it does not serve to hide with whom you are communicating (there are tools for doing this, but they involve additional complexity) and all messages encrypted with a single key can be decrypted with that key. As such keys are typically protected by a password the user must remember. It is a sufficiently accurate simplification of the process to consider the messages themselves protected by a password that the owner of the messages must remember and might possibly be forced to divulge as the fundamental limit on the security of the messages so protected. There are different tools for different purposes that exchange ephemeral keys that the user doesn’t ever know, aren’t protected by a mnemonic password, and therefore can never be forced to divulge).

These rants against PGP annoy me because PGP is an excellent tool that is marred by minor usability problems. Energy expended on ignorantly dismissing the tool is energy that could be better spent improving it.  By far the most important use cases for the vast majority of users that have any real reason to consider cryptography are only addressed by PGP.  I make such a claim based on the following:

  • Most business and important correspondence is conducted by email and despite the hyperventilation of some ignorant children, will remain so for the foreseeable future.
  • Important correspondence, more or less by definition, has a useful shelf life of more than one read and generally serves as a durable (and legally admissible) record.
  • There are people who have legitimate reasons to obfuscate their correspondents: email, even PGP encrypted email, is not a suitable tool for this task.
  • There are people who have legitimate reason to communicate messages that must not be permanently recorded and for which either the value of the communication is ephemeral or the risk is so great that destroying the archive is a reasonable trade-off: email, even with PGP, is not a suitable tool for this task.
  • There’s some noise in the rant about not being sufficient to protect against NSA targeted intercept or thwarting NSA data archiving, which makes an implicit claim that the author has some solution that might provide such protection to end users. I consider such claims tantamount to homicide.  If someone is targeted by state-level surveillance, they can’t use a Turing-complete device (any digital device) to communicate information that puts them at risk; any suggestion to the contrary is dangerous misinformation.

Current implementations of PGP have flaws:

  • For some reason, mail clients still don’t prompt for the import or generation of PGP keys whenever a new account is set up.  That’s somewhat pathetic.
  • For some reason, address books integrated into mail clients don’t have a field for the public key of the associate.  This is a bizarre omission that necessitates add-on key management plug-ins that just make the process more complicated.
  • It is somewhat complicated by IMAP, but no client stores encrypted messages locally in unencrypted form, which makes them difficult to search and reduces their value as an archival record.  This has trivial security value: your storage device is, of course, encrypted or exposing your email should your device be lost is likely to be the least of your problems.

PGP is, despite these shortcomings, one of the most important cryptographic tools available.

Awesome properties of PGP keys no other cryptographic system can touch

PGP keys are (like all cryptographic keys in use by any system) long strings of seemingly random data.  The more seemingly random, the better.  They are, by that very nature, nonmnemonic.  Public key cryptosystems, like PGP, have an awesome, incredibly useful characteristic that you can publish your public key (a long, random string of numbers) and someone you’ve never met can encrypt a message using that public key and only your private key can decrypt it.

Conversely, you can “sign” data with your “private key” and anyone can verify that you signed it by decrypting it with your public key (or more precisely a short mathematical summary of your message).  This is so secure, it is a federally accepted signature mechanism.

There’s a hypothesized attack called a Man In The Middle attack (often abbreviated “MITM”) that exploits the fact these keys aren’t really human readable (you can, but they’re so long you won’t) whereby an attacker (traditionally the much maligned Eve) intercepts messages between two parties (traditionally the secretive Alice and Bob), pretending to be Bob whilst communicating with Alice and pretending to be Alice whilst communicating with Bob.  By substituting her keys for Alice’s and Bob’s, both Bob and Alice inadvertently send messages that Eve can decrypt and she “simply” forwards Bob’s to Alice using Alice’s public key and vice versa so they decrypt as expected, despite coming from the evil Eve.

Eve must, however, be able to intercept all of Alice and Bob’s communication or her attack may be discovered when the keys change, which is not practical in the real world on an ongoing basis (but, ironically, is easier with ephemeral keys). Pretending to be someone famous is easier and could be more valuable as people you don’t know might send you unsolicited private correspondence intended for the famous person: the cure is widely disseminating key “fingerprints” to make the discovery of false keys very hard to prevent.  And if you expect people to blindly send you high-priority information with your public key, you have an obligation to mitigate the risk of a false recipient.

Occasionally it is hypothesized that this attack compromises the utility of PGP; it is a shortcoming of all cryptographic systems that the keys are not human readable if they are even marginally secure.  It is intrinsic to a public key infrastructure that the public keys must be exchanged.  It is therefore axiomatic that a PKI-based cryptographic system will be predicated on mechanisms to exchange nonmnemonic key information. And hidden key exchange, as implemented by OTR or other ephemeral key systems makes MITM attacks harder to detect.

While it is true that elliptic curve PKI algorithms achieve equivalent security with shorter keys, they are still far too long to be mnemonic.  One might nominally equivalence a 4k RSA key with a 0.5k elliptic curve key, a non-trivial factor of 8 reduction with some significance to algorithmic efficiency, but no practical difference in human readability.  Migrating to elliptic curves is on the roadmap for PGP (with GPG 2.1, now in beta) and should be expedited.

PGP Key management is a little annoying

Actually, it  isn’t so much PGP that makes this true, but rather the fact that mail clients haven’t integrated PGP into the client.  That Gmail and Yahoo mail will soon be integrating PGP into their mail clients is a huge step in the right direction even if integrating encryption into a webmail client is kind of pointless since the user is already clearly utterly unconcerned with privacy at all if they’re gifting Google or Yahoo their correspondence.  Why people who should know better still use Gmail is a mystery to me.  When people who care about data security use a gmail address it is like passing the temperance preacher passed out drunk in the gutter.  With every single message sent.  Even so, this is a step in the right direction by some good people at Google.

It is tragic that Mozilla has back-burnered Thunderbird, but on the plus side they don’t screw up the interface with pointless changes to justify otherwise irrelevant UX designers as does every idiotic change in Firefox with each release.  Hopefully the remaining community will rally around full integration of PGP following the astonishingly ironic lead of the privacy exploiting industry.

If keys were integrated into address books in every client and every corporate LDAP server, it would go a long way toward solving the valid annoyances with PGP key management; however, in my experience key management is never the sticking point, it is either key generation or the hassle of trying to deal with data rendered opaque and nearly useless by residual encryption of the data once it has reached me.

Forward Secrecy has a place.  It isn’t email.

A complaint levied against PGP that proves beyond any doubt that the complainant doesn’t understand the use case of PGP is that it doesn’t incorporate forward secrecy.  Forward secrecy is a consequence of a cryptosystem that negotiates a new key for each message thread which is not shared with the users and which the system doesn’t store.  By doing this, the correspondents cannot be forced to reveal the keys to decrypt the contents of stored or captured messages since they don’t know them.  Which also means they can’t access the contents of their stored messages because they’re encrypted with keys they don’t know.  You can’t read your own messages.  There are messaging modalities where such a “feature” isn’t crippling, but email isn’t one of them; sexting perhaps, but not email.

Indeed, the biggest, most annoying, most discouraging problem with PGP is that clients do not insert the unencrypted message into the local message store after decrypting it.  This forces the user to decrypt the message again each time they need to reference it, if they can ever find it again.  One of the problems with this is you can’t search encrypted messages without decrypting them.  No open source client I’m aware of has faced this debilitating failure of use awareness, though Symantec’s PGP desktop does (so it is solvable).  Being naive about message use wouldn’t have been surprising for the first few months of GPG’s general use, but that this failure persists after decades is somewhat shocking and frustrating.  It is my belief that the geekiness of most PGP interfaces has so limited use that most people (myself included) aren’t crippled by not being able to find PGP encrypted mail because we get so little of it.  If even a small percentage of our mail was encrypted, not ever being able to find it again would be a disaster and we’d stop using encryption.

This is really annoying because messages have the frequently intolerable drawbacks of being ephemeral without the cryptographic value of forward secrecy.

Email is normally used as a messaging modality of record.  It is the way in which we exchange contemplative comments and data that exceeds a sentence or so.  This capability remains important to almost all collaborative efforts.  The record thus created has archival value and is a fundamental requirement in many environments.  Maximizing the availability, searchability, and ease of recall of this archive is essential.  Indeed, even short form communication (“chat” in various forms), which is typically amenable to forward secrecy because of the generally low content value thus communicated, should have the option of PGP encryption instead of just OTR in order to create a secure but archival communications channel.

A modest proposal

I’ve been using PGP since the mid 1990s.  I have a key from early correspondence on PGP from 1997 and mine is from 1998.  Yet while I have about 2,967 contacts in my address book I have only 139 keys in my GPG keyring.  An adoption rate of 4.7% for encrypted email isn’t exactly a wild success.  I don’t think the problems are challenging and while I very much appreciate the emergence of cryptographically secure communications modalities such as OTR for chat and ZRTP for voice, I’ve been waiting for decades for easy-to-use secure email.  And yet, when people ask me to help them set up encrypted email, I generally tell them it is complicated, I’m willing to help them out, but they probably won’t end up using it.  Over the years, a few relatively easy to fix issues have retarded even my own use:

  • The fact that users have to find and install a somewhat complex plugin to handle encryption is daunting to the vast majority of users.  Enigmail is complicated enough that it is unusable without in-person walk-through support for most users.  Even phone support doesn’t get most people through setup. Basic GPG key generation and management should be built into the mail client.  Every time one sets up a new account, you should have to opt out of setting up a public key and there’s no reason for any options by default other than entering a password to protect the private key.
  • Key fields should be built into the address book of every mail client by default.  Any mail client that doesn’t support a public key field should be shamed and ridiculed.  That’s all of them until Gmail releases end-to-end as a default feature, though that may never happen as that breaks Google’s advertising model.  Remember, Google pays all their developers and buys them all lunch solely by selling your private data to advertisers.  That is their entire business model. They do not consider this “evil,” but you might.
  • I have no idea why my received encrypted mail is stored encrypted on my encrypted hard disk along with hundreds of thousands of unencrypted messages and tens of thousands of unencrypted documents.  Like any sensible person who takes a digital device out of the house (or leaves it unprotected in the house), I encrypt my local storage to protect those messages and documents from theft and exploitation.  My encrypted email messages are merely data cruft I can’t make much use of since I can’t search for them.  That’s idiotic and cripples the most important use modality of email: the persistent record. Any mail client should permanently decrypt the local message store unless the user specifically requests a message be stored encrypted, an option that should be the same for a message that arrived encrypted or unencrypted.
  • Once we solve the client storage failure and make encrypted email useful for something other than sending attachments (which you can save, ZOMG, in unencrypted form) and feeling clever for having gotten the magic decoder ring to work, then it would make sense to modify mail servers to encrypt all unencrypted incoming mail with the user’s public key, which mitigates a huge risk in having a mail server accessible on the internet: that the historical store of data there contained is remotely compromised.  This protects data at rest (data which is often, but not assuredly, already protected in transit by encrypted transport protocols.)  End-to-End encryption using shared public keys is still optimal, but leaving the mail store unencrypted at rest is an easily solved security failure and in protection in transit is largely solved (and would be quickly if gmail bounced any SMTP connection not protected by TLS 1.2+.)

Fixing the obvious usability flaws in encrypted email are fairly easy.  Public key cryptography in the form of PGP/GPG is an incredibly powerful and tremendously useful tool that has been hindered in uptake by limitations of perception and by overly stringent use cases that have created onerous limitations.  Adjusting the use model to match requirements would make PGP far more useful and far easier to convince people to use.

Phil Zimmerman’s essay “why I wrote PGP” applies today as much as it did in 1991:

What if everyone believed that law-abiding citizens should use postcards for their mail? If a nonconformist tried to assert his privacy by using an envelope for his mail, it would draw suspicion. Perhaps the authorities would open his mail to see what he’s hiding.

It has been almost 25 years and never has the need for universally encrypted mail been more obvious.  It is time to integrate PGP into all mail clients.

 

Posted at 14:39:09 GMT-0700

Category: FreeBSDLinuxtechnology

TLS 1.0 Hatin’ the Game

Wednesday, June 1, 2011 

After much reading and interpreting, it became clear there was no more advice for configuration variations to get client cert login working. It seemed Chrome was doing it right, IE not even trying, and Firefox failing. No advice as to why and setting LogLevel to debug didn’t add much in the way of useful hints.

TLS_bad.JPG

Jared Davenport, for reasons that would never have occurred to me, tried turning off TLS 1.0 in firefox as an allowed protocol. PCI compliance requires turning off a bunch of weaker/compromised protocols and ciphers anyway, so I already had:

SSLProtocol -ALL +SSLv3 +TLSv1

A quick test of

SSLProtocol -ALL +SSLv3

solved the problem with firefox. IE still refuses to talk to SSL, but IE is a stupidhead anyway. OK, it annoys me as the same client cert works on CACert.org’s site so something there is working right that isn’t on my box, but as I never use IE, I think I can let it go

no_tls_good.JPG
Posted at 01:21:25 GMT-0700

Category: FreeBSDLinux

How to Disable CTL-Return in Thunderbird

Thursday, September 23, 2010 

One of the stupidest keyboard shortcuts I’ve run into is Thunderbirds CTRL-Return automatic send function. Maybe I type sloppy, but I frequently CTRL-V to paste a link into a message and hit return just a little too fast to continue typing and, damn it, the embarrassing, incomplete message is gone.

It turns out I’m not the only one. I found this great link
http://blogs.sun.com/LetTheSunShineIn/entry/changing_thunderbird_keyboard_shortcut

which has, itself, a link to a pretty cool plugin that lets you remap the keyboard shortcuts.
http://mozilla.dorando.at/keyconfig.xpi

But it does not (at least with Thunderbird 3.1.4 on window) list the dreaded ctrl-enter stupidkey. Now windows 7 search is astonishingly stupid (how come windows, 20 years on, still can’t give a marginal search function when back in 1990 OnLocation could return every file on my Mac, including searching by content, in a few milliseconds? Progress my ass) but I found the right “prefs.js” (eventually) at C:\Users\dgessel\AppData\Roaming\Thunderbird\Profiles\mwwkrsno.default.

As I’d modified a few keyboard commands with keyconfig already, prefs.js had a nice friendly indicator of where I should insert my own guerrilla modification (about the middle of the file) and there I pasted in
user_pref(“keyconfig.main.key_send”, “!][][“);
and when I launched Thunderbird, ctrl-enter was disabled. YAY!

(The following message was a “note to self” – I typed ctrl-enter and…)

Yep. Message still here… doesn’t work.

(…noted that the message was not sent thus ctrl-enter no longer works.  The fix, therefore, does work.)

If you want to customize your experience, there’s a nice command reference here
http://kb.mozillazine.org/Keyconfig_extension:_Thunderbird

I added CTL-ALT-RETURN as “send later” which I don’t think I’ll hit accidentally.

Posted at 23:33:40 GMT-0700

Category: Linuxtechnology