linux

Mobotix Notifier in Python – get desktop messages from your cameras

Tuesday, June 6, 2023 

I wrote a little code in python to act as a persistent, small footprint LAN listener for Mobotix cameras IP Notify events.  If such a thing is useful to you, the code and a .exe compiled version are linked/inline.  It works on both Windows and Linux as python code.  For Windows there’s a humongous (14MB) .exe file use if you don’t want to install Python and mess with the command line in power shell.

Message generated by the Windows Notifier

Mobotix cameras have a pretty cool low-level feature by which you can program via the camera web interface a raw IP-packet event to send to a destination if the camera detects a trigger, for example motion, PIR over threshold, noise level, thermal trigger, or the various AI detectors available on the 7 series cameras. Mobotix had a simple notification application, but some of these older bits of code aren’t well supported any more and Linux support didn’t last long at the company, alas.  The camera runs Linux, why you’d want a client appliance to run anything but Linux is beyond me, but I guess companies like to overpay for crappy software rather than use a much better, free solution.

I wanted something that would push an otherwise not intrusive notification when the camera triggered for something like a cat coming by for dinner, pushing a desktop notification.  Optimally this would be done with broadcast packets over UDP, but Mobotix doesn’t support UDP broadcast IP Notify messaging yet, just TCP, so each recipient address (or DNS name) has to be specified on each camera, rather than just picking a port and having all the listeners tune into that port over broadcast.  Hopefully that shortcoming will be fixed soon.

This code runs headless, there’s no interaction.  From the command line just ./mobotix_notifier.py & and off it goes.  From windows, either the same for the savvy or double click the exe.  All it does is listen on port 8008/TCP and if it gets a message from a camera, reach out and grab the current video image, iconify it, then push a notification using the OS’s notification mechanism which appears as a pop-up window for few seconds with a clickable link to open the camera’s web page.  It works if you have one or a 100 cameras, but it is not intended for frequent events which would flood the desktop with annoyance, rather a front door camera that might message if someone’s at the door.  In a monitoring environment, it might be useful for signaling critical events.

Mobotix Camera Set Up

On the camera side there are just two steps: setting up an IP-Notify action from the Admin Menu and then defining an Action Group from the Setup Menu to trigger it.

IP Notify Profile

The title is the default “SimpleNotify” – that can be anything.

The Destination addresses are the IPs of the listener machines and port numbers.  You can add as many as needed but for now it is not possible to send a UDP broadcast message as UDP isn’t supported yet.  It may be soon, I’ve requested the capability and I expect the mechanism is just a front end for netcat (nc) as it would be strange to write a custom packet generator when netcat is available.  For now, no broadcast, just IP to IP, so you have to manually enumerate all listeners.

I have the profile set for sequential send to all rather than parallel just for debugging, devices further down the list will have lower latency with parallel send.

The data protocol is raw TCP/IP, no UDP option here yet…

The data type is plain text, which is easier to parse at the listener end.   The data structure I’m using reads: $(id.nam), $(id.et0) | Time: $(fpr.timestamp) | Event: $(EVT.EST.ACTIVATED) | PIR: $(SEN.PIR) | Lux: $(SEN.LXL) | Temp: $(SEN.TOU.CELSIUS) | Thermal: $(SEN.TTR.CELSIUS) but it can be anything that’s useful.

Mobotix cameras have a robust programming environment for enabling fairly complex “If This Then That” style operations and triggering is no exception.  One might reasonably configure the Visual Alarm (now with multiple Frame Colors, another request of mine, so that you can have different visual indicators for different detected events, create different definitions at /admin/Visual Alarm Profiles), a fairly liberal criterion might be used to trigger recording, and a more strict “uh oh, this is urgent” criterion might be used to trigger pushing a message to your new listeners.

Action Group Push Message

This config should be fairly obvious to anyone familiar with Mobotix camera configuration: it’s configured to trigger at all detected events but not more than once every 5 seconds.  given it is pushing a desktop alert, a longer deadtime might be appropriate depending on the specifics of triggering events that are configured.

That’s all that’s needed on the camera end: when a triggering event occurs the camera will take action by making a TCP connection to the IP address enumerated on the selected port and, once the connection is negotiated push the text structure.  All we need now is something to listen.

Python Set Up

The provided code can be run as a python “application” but python is an interpreted language and so needs the environment in which to interpret it properly configured.  I also provide a compiled exe derived from the python code using PyInstaller, which makes it easier to run without Python on Windows where most users aren’t comfortable with command lines and also integrates more easily with things like startup applications and task manager and the like.

If you’re going to run the python command-line version, you can use these instructions for Windows, or these for Linux to set up Python. Just make sure to install a version more recent than 3.7 (you’d have to work at installing an older version than that).  Then, once python is installed and working, install the libraries this script uses in either windows powershell or Linux shell as below.  Note that python3 specifies the 3.x series of python vs. 2.x and is only necessary in systems with earlier version baggage like mine.

python[3] -m pip install plyer dnspython py-notifier pillow --upgrade

Once python is installed, you should be able to run the program from the directory by just typing ./mobotix_notifier.py, obviously after you’ve downloaded the code itself (see below).

Firewalls: Windows and Linux

Linux systems often have Uncomplicated Firewall (UFW) running.  The command to open the ports in the firewall to let any camera on the LAN reach the listener is:

sudo ufw allow from 192.168.100.0/24 proto tcp to any port 8008
# if you make a mistake
sudo ufw status numbered
sudo ufw delete 1

This command allows TCP traffic in from the LAN address (192.168.100.0/24, edit as necessary to match your LAN’s subnet) on port 8008.  If a broadcast/UDP version comes along, the firewall rule will change a little.  You can also reduce the risk surface by limiting the allowed traffic to specific camera IPs if needed.

On windows, the first time the program is run, either the python script or the executable, you’ll get a prompt like

Windows Defender Notification for Mobotix Notifier

You probably don’t need to allow public networks, but it depends on how you’ve defined your network ranges whether Windows considers your LAN public or private.

Default Icon Setup

One of the features of the program is to grab the camera’s event image and convert it to the alert icon which provides a nearly uselessly low rez visual indicator of the device reporting and the event that caused the trigger.  The icon size itself is 256×256 pixels on linux and 128×128 on windows (.ico).  Different window managers/themes provide more or less flexibility in defining the alert icons.   Mine are kinda weak.

Linux event notificationThe win-10 notification makes better use of the icon.  Older versions of linux had a notification customization tool that seems to have petered out at 16.x, alas.  But the icons have some detail if your theme will show them.

Another feature is that the code creates the icon folder if it doesn’t exist.  It almost certainly will on Linux but probably won’t on windows unless you’ve run some other Linuxy stuff on your windows box.  The directory created on windows is your home directory\.local\share\icons\. On Linux systems, the directory should exist and is ~/.local/share/icons/. In that directory you should copy the default camera icon as “mobotix-cam.ico” like so:

where to put mobotix-cam.ico

You can put any icon there as your preferred default as long as it is in .ico format, or use the one below (right-click on the image or link and “save as” to download the .ico file with resolution layers):

Mobotix Camera M16If, for some reason, the get image routine fails, the code should substitute the above icon so there’s a recognizable visual cue of what the notification is about.

mobotix_notifier.py code

The python code below can be saved as “mobotix_notifier.py” (or anything else you like) and the execution bit set, then it can be run as ./mobotix_notifier.py on Linux or python .\mobotix_notifier.py on Windows. On Linux, the full path to where you’ve installed the command can be set as a startup app and it will run on startup/reboot and just listen in the background.  It uses about 13 seconds a day of CPU time on my system.

Click to download the Windows .exe which should download as mobotix_notifier.exe. (14.0MiB)  After the above configuration steps of on the camera(s) and firewall are completed it should start silently and run in the background after launch (kill it with task manager if needed) and push desktop alerts as expected.  I used “UC” alarms to test rather than waiting for stray cats.

The python code is:

#!/usr/bin/env python3

import requests
from PIL import Image
import socket
from plyer import notification
import io
import os.path

# note windows version needs .ico files
# note windows paths have to be r type to handle
# backslashes in windows paths
# Check operating environment and define path names
# for the message icons accordingly.
# if OS path doesn't exist, then create it.

if os.name == "nt":
    Ipath = r"~\.local\share\icons\mobotix-cam.ico"
    Epath = r"~\.local\share\icons\mobotix-event.ico"
    fIpath = os.path.expanduser(Ipath)
    fEpath = os.path.expanduser(Epath)
    dirpath = os.path.dirname(fEpath)
    if not os.path.exists(dirpath):
        os.makedirs(dirpath)

else:
    Ipath = "~/.local/share/icons/mobotix-cam.png"
    Epath = "~/.local/share/icons/mobotix-event.png"
    fIpath = os.path.expanduser(Ipath)
    fEpath = os.path.expanduser(Epath)
    dirpath = os.path.dirname(fEpath)
    if not os.path.exists(dirpath):
        os.makedirs(dirpath)


def grab_jpeg_image(camera_ip):
    """Grabs a JPEG image from the specified camera IP."""

    # Make a request to the camera IP
    response = requests.get(f"http://{camera_ip}/control/event.jpg", stream=True) # noqa

    # Check if the request was successful
    if response.status_code == 200:
        # Convert the response data to an image
        image = Image.open(io.BytesIO(response.content))

        # Return the image
        return image

    else:
        # import the default icon
        image = Image.open(fIpath)

        # Return the image
        return image


def convert_jpeg_to_png(image, width, height):
    """Converts a JPEG image to a PNG image."""

    # size = width, height

    # Scale the image
    image.thumbnail((width, height), Image.Resampling.LANCZOS)

    # Save the image according to OS convention
    if os.name == "nt":
        icon_sizes = [(16, 16), (32, 32), (48, 48), (64, 64), (128, 128)]
        image.save(fEpath, format='ICO', sizes=icon_sizes)
    else:
        image.save(fEpath)


def iconify(src_ip):

    # Grab the JPEG image from the camera
    image = grab_jpeg_image(src_ip)

    # Convert the JPEG image to a PNG image
    convert_jpeg_to_png(image, 256, 256)


def reverse_dns_lookup(src_ip):

    try:
        return socket.gethostbyaddr(src_ip)[0]
    except socket.gaierror:
        return "no dns"
    except socket.herror:
        return "no dns"


def test_str(answer):
    try:
        return str(answer)
    except TypeError:
        return answer.to_text()


def listener():
    """Listens for incoming connections on port 8008."""

    # Create a socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

    # Bind the socket to port 8008
    sock.bind(("0.0.0.0", 8008))

    # Listen for incoming connections
    sock.listen(1)

    while True:
        # Accept an incoming connection
        conn, addr = sock.accept()

        # Receive the payload of the packet
        data = conn.recv(2048)

        # Close the connection
        conn.close()

        # convert from literal string to remove b' prefix of literal string
        data = str(data)[2:-1]

        # Extract the source IP from the address
        src_ip = addr[0]

        # Grab the event image as an icon
        iconify(src_ip)

        # Do a DNS lookup of the source IP
        answer = reverse_dns_lookup(src_ip)

        # Get the hostname from the DNS response
        hostname = test_str(answer)

        # Write the hostname to notify-send

        title = (f"Event from: {hostname} - {src_ip}")
        message = (f"{data} http://{src_ip}/control/userimage.html")

        notification.notify(
            title=title,
            message=message,
            app_icon=fEpath,
            timeout=30,
            toast=False)

        # Echo the data to stdout for debug
        # print(f"Event from {hostname} | {src_ip} {data}")


if __name__ == "__main__":
    listener()

Please note the usual terms of use.

Posted at 08:21:09 GMT-0700

Category: CodeHowToLinuxTechnology

Tagging MP3 Files with Puddletag on Linux Mint

Tuesday, March 23, 2021 

A “fun” part of organizing an MP3 collection is harmonizing the tags so the datas work consistently with whatever management schema you prefer.  My preference is management by the file system—genre/artist/year/album/tracks works for me—but consistent metainformation is required and often disharmonious.  Finding metaharmony is a chore I find less taxing with a well structured tag editor and to my mind the ur-meta-tag manager is  MP3TAG.

The problem is that only works with that dead-end spyware riddled failing legacyware called “Windows.” Fortunately, in Linux-land we have puddletag, a very solid clone of MP3TAG.  The issues is that the version in repositories is (as of this writing) 1.20 and I couldn’t find a PPA for the latest, 2.0.1.  But compiling from source is super easy and works in both Linux Mint 19 and Ubuntu 20.04 and version  2.20 on 22.04 which contains my mods to latinization of foreign scripts (yay open source!):

  1. Install pre-reqs to build (don’t worry, if they’re installed, they won’t be double installed)
  2. get the tarball of the source code
  3. expand it (into a reasonable directory, like ~/projects)
  4. switch into that directory
  5. run the python executable “puddletag” directly to verify it is working
  6. install it
  7. tell the desktop manager it’s there – and it should be in your window manager along with the rest of your applications.

The latest version as of this post was 2.0.1 from https://github.com/puddletag/puddletag

sudo apt install python3-pyqt5 python3-pyqt5.qtsvg python3-pyparsing python3-mutagen python3-acoustid libchromaprint-dev libchromaprint-tools libchromaprint1 
wget href="https://github.com/puddletag/puddletag/releases/download/2.0.1/puddletag-2.0.1.tar.gz 
tar -xvf puddletag-2.0.1.tar.gz
cd puddletag-2.0.1/ 
cd puddletag 
./puddletag 
sudo python3 setup.py install 
sudo desktop-file-install puddletag.desktop

A nice feature is the configuration directory is portable and takes your complete customization with you – it is an extremely customizable program so you can generally configure it as fits your mental model.  Just copy the entire puddletag directory located at ~/.configure/puddletag.

Posted at 15:19:01 GMT-0700

Category: AudioHowToLinuxPositiveReviews

10 Gbyte Win10 Spyware “upgrade” now forced on users

Sunday, September 27, 2015 

Microsoft has, historically, done some amazingly boneheaded things like clippy, Vista, Win 8, and Win 10.  They have one really good product: Excel, otherwise everything they’ve done has succeeded only through illegal exploitation of an aggressively defended monopoly. OK, maybe the Xbox is competitive, but I’m not much of a gamer.

Sadly for the world, the model of selling users for profit to advertisers and spies has gained ground to the point where Microsoft was starting to look like the least evil major entity in closed-source computing.  Poor microsoft.  To lose the evil crown must be at least as humiliating as their waning revenue and abject failures in the mobile space (so strange… try to enter a space where they don’t have a monopoly to force users to accept their mediocre crap and they fail, who’da thunk it?)

“There is a difference between policy and practice. We don’t read customers mail. We don’t read customer documents. We don’t triangulate YouTube views and searches. We don’t use the content of your Hotmail to target ads in Bing,”

Frank Shaw, Corporate Vice President of Corporate Communications for Microsoft

Well, never fear: Windows 10 is here and they’re radically one-upping the data theft economy by p0wning not just the data you idiotically entrust to someone else’s server for free without ever considering why they’re giving you that useful service for “free” or what they, or whoever buys their ultimately failed business, might do with your data, but also the data you consider too sensitive for the Google or the Apple.  Windows 10 exfiltrates all your data to Microsoft for their use and profit without your information.  Don’t believe it? Read their Privacy Statement.

Finally, we will access, disclose and preserve personal data, including your content (such as the content of your emails, other private communications or files in private folders), when we have a good faith belief that doing so is necessary.

And it is free (as in beer but not as in speech).  What could possiblay go wrong?

Well, people weren’t updating fast enough so Microsoft is now pushing that update on you involuntarily.  Do you have a data cap that a 10G download might break and cost you money?  So what!  Your loss!  Don’t have enough space on your drive for a 10G hidden folder of crapware foisted off on you without your permission?  Tough crap, Microsoft don’t care.

To be clear, Windows 10 is spyware.  If this was coming from a teenage hacker somewhere, they’d be facing jail time.  It is absolutely, unequivocally malware that will create a liability for you if you use it.  If you have any confidentiality requirement, you must not install windows 10.  Ever. Not even on your home machine.  Just don’t.

The only way to prevent this is really annoying and a little risky: disable automatic downloads.  One of the problems with Microsoft’s operating systems is the unbelievably crappy spaghetti code that results in a constant flow of cracks, a week’s worth are patched every Tuesday.  About 1 serious vulnerability every fortnight these days (note this is about the same as Ubuntu and about 1/4 the rate of OSX or iOS, why people think Apple products are “secure” is beyond me – live in that fantasy walled garden!  But nice logo you paid a 50% premium for on your shiny device). Not patching increases the risk that some hacker somewhere will steal your datas, but patching guarantees that Microsoft will steal your datas.  Keep your anti-virus up to date and live a little dangerously by keeping Microsoft out.

Here’s an interesting article: how-to-clean-the-windows-10-crapware-off-your-windows-7-or-81-pc

And a tool referenced in that article: GWX control panel (that can help remove the windows 10 infection if you got it).

And a list of patches I found that are related to Win10 malware that you can remove if you haven’t installed it yet (Windows 10 eliminates the ability to choose or selectively remove patches, once you’re in for the ride, you’re chained in: all or nothing.)

Basic advice:

  • Disable automatic updates and automatic downloads of updates.
  • Review each update Microsoft offers.  This is tedious, my win 7 install reports 384 updates, 5-10 a week, but other than security patches, you probably don’t really need them.  Only install a patch if there’s a reason.  Sorry, that sucks, but there’s always Linux Mint: free like beer AND free like speech.
  • If you’re still on Win 7/8, uninstall the spyware Microsoft has probably already installed.  If you’re on Windows 8, you probably want to upgrade to Windows 7 if at all possible.
  • If you succumbed to the pressure and became a Microsoft Product by installing Windows 10, uninstall it.
  • If uninstall doesn’t work, switch to Mint or reinstall 7.

Most importantly, if you develop software for servers or for end users, stop developing for Microsoft (and Apple too).  Respect the privacy of your customers by not exposing them to exploitation by desperate operating system vendors.  In many classes of applications, your customers buy their computers to run your software: they don’t care what operating system it requires – that should be transparent and painless.  Microsoft is no longer an even remotely acceptable choice.  Server applications should run under FreeBSD or OpenBSD and desktop applications should run under Linux.  You can charge more and generate more profit because the total net cost for your customers will be lower.  Split the difference and give them a more reliable, more secure, and lower cost environment and make more money doing so.

Posted at 08:07:54 GMT-0700

Category: FreeBSDHowToLinuxSecurityTechnology

Successful connect to WPA2 with Linux Mint 17

Saturday, September 26, 2015 

I found myself having odd problems connecting to WPA2 encrypted wireless networks with a new laptop.  There must be more elegant solutions to this problem, but this worked for me.  The problem was that I couldn’t connect to a nearby hotspot secured with WPA2 whether I used the default config tool for mint, Wicd Network Manager, or the command line.  Errors were either “bad password” or the more detailed errors below.

As with any system variation mileage may vary, my errors look like:

wlan0: CTRL-EVENT-SCAN-STARTED 
wlan0: SME: Trying to authenticate with 68:72:51:00:26:26 (SSID='WA-bullet' freq=2462 MHz)
wlan0: Trying to associate with 68:72:51:00:26:26 (SSID='WA-bullet' freq=2462 MHz)
wlan0: Associated with 68:72:51:00:26:26
wlan0: CTRL-EVENT-DISCONNECTED bssid=68:72:51:00:26:26 reason=3 locally_generated=1

and my system config is reported as:

# lspci -vv |grep -i wireless
3e:00.0 Network controller: Intel Corporation Wireless 7260 (rev 6b)
 Subsystem: Intel Corporation Dual Band Wireless-AC 7260
# uname -a
Linux dgzb 3.16.0-38-generic #52~14.04.1-Ubuntu SMP Fri May 8 09:43:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

I found useful commands for manually setting up a wpa_supplicant.conf file here, and for disabling 802.11n here. The combination was needed to get things working.

The following successfully connects to a WPA2-secured network:

$ sudo su
$ iw dev
 ... Interface [interfacename] (typically wlan0, assumed below)
$ iw wlan0 scan
 ... SSID: [ssid]
 ... RSN: (if present means the network is secured with WPA2)
$ wpa_passphrase [ssid] >> /etc/wpa_supplicant.conf 
...type in the passphrase for network [ssid] and hit enter...
$ sh -c 'modprobe -r iwlwifi && modprobe iwlwifi 11n_disable=1'
$ wpa_supplicant -i wlan0 -c /etc/wpa_supplicant.conf

(should show CTRL-EVENT-CONNECTED)
(open a new terminal leaving the connection open, ending the command disconnects)

$ sudo su
$ dhclient wlan0

(should be connected now)

Posted at 10:16:28 GMT-0700

Category: HowToLinuxTechnology

Windows 10 Privacy Annihilator

Tuesday, August 4, 2015 

Why would Microsoft, a company whose revenue comes entirely from sales of Windows and Office, start giving Windows 10 away – not just giving it away, but foisting it on users with unbelievably annoying integrated advertisements in the menu of Win 7/8 that pop up endlessly and are tedious to remove and reinstall themselves constantly?

Have they just gone altruistic?  Decided that while they won’t make software free like speech, they’ll make it free like beer? Or is there something more nefarious going on? Something truly horrible, something that will basically screw over the entire windows-using population and sell them off like chattel to any bidder without consent or knowledge?

Of course, it is the latter.

Microsoft is a for-profit company and while their star has been waning lately and they’ve basically ceded the evil empire mantle to Apple, they desperately want to get into the game of stealing your private information and selling it to whoever is willing to pay.

So that’s what Windows 10 does.  It enables Microsoft to steal all of your information, every email, photo, or document you have on your computer and exfiltrate it silently to Microsoft’s servers, and to make it legal they have reserved the right to give it to whoever they want.  This isn’t just the information you stupidly gifted to Google by being dumb enough to use Gmail or ignorantly gifted to Apple by being idiotic enough to load into the iButt, but the files you think are private, on your computer, the ones you don’t upload.  Microsoft gets those.

Finally, we will access, disclose and preserve personal data, including your content (such as the content of your emails, other private communications or files in private folders), when we have a good faith belief that doing so is necessary.

They’ll “access” your data and “disclose” it (meaning to a third party) whenever they have a good faith belief that doing so is necessary.  No warrant needed.  It is necessary for Microsoft to make a buck, so if a  buck is offered for your data, they’re gonna sell it.

If you install Windows 10, you lose. So don’t. If you need to upgrade your operating system, it is time to switch to something that preserves Free like speech: Linux Mint is probably the best choice.

If you’re forced to run Windows 10 for some reason and can’t upgrade to windows 7, then follow these instructions (and these) and remain vigilant, Microsoft’s new strategy is to steal your data and sell it via any backdoor they can sneak past you. Locking them down is going to be a lot of work and might not be possible so keep an eye out for your selfies showing up on pr0n sites: they pay for pix and once you install Windows 10, Microsoft has every right to sell yours.


 

Update: you can’t stop windows 10 from stealing your private data

That’s not quite true – if you never connect your computer to a network, it is very unlikely that Microsoft will be able to secretly exfiltrate your private data through the Windows 10 trojan.  However, it turns out that while the privacy settings do reduce the amount of data that gets sent back to Microsoft, they continue to steal your data even though you’ve told them not to.

Windows 10 is spyware.  It is not an operating system, it is Trojan malware masquerading as an operating system that’s true purpose is to steal your data so Microsoft can sell it without your consent.  If you install Windows 10, you are installing spyware.

Win 10 has apparently been installed 65 million times.  That’s more than 3x as many users’ most intimate, most private data stolen as by the Ashley Madison attack.  If you value privacy, if the idea that you might be denied a loan or insurance because of secret data stolen from your computer without your consent bothers you, if the idea of having evidence of your potential crimes shared with law enforcement without your knowledge and without a warrant worries you then do not install windows 10.  Ever.

Posted at 11:00:30 GMT-0700

Category: PrivacyTechnology

Copying Text Without the Horrible Formatting

Saturday, August 16, 2014 

Have you ever copied some text off a web page or a document and then gone to paste it in another document or spreadsheet only to find some horribly formatted hypertext pasted in for some bizarre reason, then had to go through the hassle of trying to figure out how to remove the formatting?

Have you ever used Putty or another SSH client that automatically copies highlighted text to the copy buffer and allows pasting with a middle click and wished all programs were this smart?

Has anyone, ever, in the history of using a computer, WANTED to paste formatted text from a web page or drop some idiotic OLE object into their FrameMaker document?  I know I’ve never once wanted that to happen.

Tonight I had to copy a 100 or so mac addresses out of a DHCP list from the web interface of pfSense into an Excel table and each damn time I got stupid formatting and then had to select the cell, select the drop down menu for paste options, select paste as text, repeat.  Holy crap, what the hell were they thinking?  No clue.

None of the paste solutions recommended for Excel worked for me and OpenOffice/Libre were just as screwed up.  But I found some solutions for the copy side for Windows.  Some of the plugins should work on Linux.  If you’re using a Mac, The Steve has already decided how your work is permitted to look and the Apple goons will probably break your fingers if you try to modify formatting.

  • Auto Copy makes Chrome on windowz almost as efficient as a linux application! Copy as text, select to copy. Middle click to paste.  Dang. But it doesn’t seem to always remove formatting (select to copy works reliably though).
  • Copy as Plain text fixes this stupidity on Firefox.
  • UPDATE: Márton Anka is an awesome developer who writes some of the best code on the internet and his plugin PLAINCOPY, is an excellent solution.
  • Autocopy2 adds the incredibly useful select to copy to Firefox.  Once you get used to it, you’ll be frustrated with applications that don’t support it.
  • This edit to maker.ini will prefer pasting plain text (or now UTF8) over OLE2, eliminating that horror from FrameMaker.

It turns out there’s a universal solution for Windows.

  • PureText removes formatting from text on the clipboard and pastes it with an alternate key command (like Windows-V), so even copying from word documents to excel isn’t a horrible nightmare of tedium.

I haven’t yet figured out how to copy images from Firefox to Thunderbird without pasting it as a reference to the original image.  Pasting an HTML reference to remote content means the recipient either doesn’t see the image (because they don’t auto-load remote content or because they don’t have permission to load it or aren’t on-line when they read their mail) or Thunderbird makes a request to the referenced site to load the media creating a privacy violating log entry.  The most convenient solution I’ve found is to paste the image into irfanview first and then copy from there into Thunderbird.

Posted at 15:40:40 GMT-0700

Category: HowToTechnology

Moar Privacy

Thursday, December 9, 2010 

I’m using an Ubuntu VM for private browsing, and like many people, I’m stuck using a mainstream OS for much of my work (Win7) due to software availability constraints. But some software works much better in a linux environment and Ubuntu is as pretty as OSX, free, and installs easily on generic x86 hardware.

It is also pretty straightforward to install an isolated and secure browsing instance using VirtualBox. It takes about 20G of hard disk and will use up at least 512K (better 1G) of your system RAM. If you want to run this sort of config, your laptop should have more than enough disk space and RAM to support the extra load without bogging, but it is a very solid solution.

Installing Ubuntu is easy – even easier with an application like VirtualBox – just install virtualbox, download the latest ubuntu ISO, and install from there. If you’re on bare metal, the easiest thing to do is burn a CD and install off that.

Ubuntu desktop comes with Firefox in the tool bar. Customizing for private browsing is a bit more involved.

My first steps are to install:

NoScript is an easy win. It is a bit of a pain to set up at first, but soon you add exceptions for all your favorite sites and while that isn’t great security practice, it is essential for sane browsing. NoScript is particularly helpful when browsing the wacky parts of the net and not getting exotic browsing diseases: it is your default dental dam. Be careful of allowing domains you don’t recognize – Google them first and make sure you understand why they need to run a script on your computer and that it is safe. A lot of sites use partners for things like video feeds, so if some function seems broken, you probably need to allow that particular domain. On the other hand, most of the off-site scripts are tracking or stats and you really don’t need to play along with them.

BetterPrivacy is a new one for me. I am very impressed that it found approximately 1.3 zillion (OK 266) different company flash cookies AFTER I had installed TACO and noscript etc. You bastards. I’m sure I can enjoy hulu without making my play history shared-available to every flash site I might visit. Always Sunny in Philadelphia marks me as a miscreant. I flush the flash cookies on starting silently (preferences).

TACO is a bit intrusive, but it seems to work to selectively block tracking and advertising cookies. At least the pop up is comforting. For private browsing, I’d set it to reject all classes of tracking cookies (change the preferences from default).

User Agent Switcher is useful when you’re deviating from the mainstream. Running Ubuntu pretty much flags you as a trouble maker or at least a dissident. Firefox maybe a bit less so, but you are indicating to advertisers that you don’t respect the expertise of those people far smarter than you who pre-installed IE (or Safari) to make your life easier. Set your user agent to IE 8 because the nail that sticks up gets pounded down.

Torbutton needs Tor to work. Tor provides really good privacy, but is a bit involved. The Tor Button Plugin for firefox makes it seem easier than it really is: you install it and click “use tor” and it looks like it is working but the first site you visit you get an proxy error because Tor isn’t actually running (DOH!).

To get Tor to work, you will have to open a terminal and do some command line fu before it will actually let you browse. Tor is also easier to install on Ubuntu than on Windows (at least for me, but as my browser history indicates I’m a bit of a miscreant dissident, so your mileage may vary).

Starting with these fine instructions.

sudu gedit /etc/apt/sources.list
add
deb http://deb.torproject.org/torproject.org/ lucid main
deb-src http://deb.torproject.org/torproject.org/ lucid main

Then run
gpg --keyserver keys.gnupg.net --recv 886DDD89
gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -
sudo apt-get update
sudo apt-get upgrade
sudo apt-get update
sudo apt-get install tor tor-geoipdb

Install vidalia with the graphical ubuntu software center or with
sudo apt-get install vidalia

Tor expects Polipo. And vidalia makes launching and checking on Tor easier, so remove the startup scripts. (If Tor is running and you try to start it from vidalia, you get an uninformative error, vidalia has a “launch at startup” option, so let it run things.) Vidalia appears under the Applications->Network.

sudo update-rc.d -f tor remove

Polipo was installed with Tor, so configure it:
sudo gedit /etc/polipo/config

Clear the file (ctrl-a, delete)
paste in the contents of this file:

UPDATE: paste in the contents of this file:

(if the link above fails, search for “polipo.conf” to find the latest version)

I added the binary for polipo in Vidalia’s control panel, but that may be redundant (it lives in /usr/bin/polipo).

I had to reboot to get everything started.

And for private chats, consider OTR!

Posted at 17:45:45 GMT-0700

Category: PoliticsTechnology

IBM BoMC on the Fritz

Sunday, September 19, 2010 

I was updating the firmware of our servers with IBM’s Bootable Media Creator – it is usually a great tool: it builds a bootable linux disk with all the latest firmware patches for your system – or all IBM systems if you want with a small utility that fetches all the latest updates for you and assembles them into an .iso.

Attached Message Part

The current version is ibm_utl_bomc_2.10_windows_i386.exe, but when you run it, it fails when it gets to uncompressing ibm_utl_boot_tools-130_anyos_x86-64-mid.zip (probably ibm_utl_boot_tools-130_anyos_x86-64-full.zip too, if you need that for your system instead). I finally noticed v130 was only 4mb and my old V110 was 65MB.

The util only downloads about 500k of each binary patch each time it is run. Fortunately, the efforts are cumulative. Unfortunately any driver bigger than about 1MB is effected and will not download completely the first time. Or the first 2x(size in mb) times. If you run ibm_utl_bomc_2.10_windows_i386.exe -m 8863 -l C:\temp from the command line (as an example, assuming your machine type is 8863) TWICE and the second time you don’t see something like:
(1 of 8) Acquiring ibm_fw_diag_zuyt38a_linux_i386...
Already downloaded.

for any driver, then it isn’t fully downloaded. Keep repeating. I wrote a script to automate the process and put about 100 repetitions of the command in the batch and went out to dinner. When I got back, all the drivers were reporting “already downloaded.”

@echo off
SET LOOP=0
ibm_utl_bomc_2.10_windows_i386.exe -m 8863 -l C:\temp
[copy and paste this or use a do loop to repeat 100x]
:END

Posted at 21:23:22 GMT-0700

Category: LinuxTechnology

Mulberry Mail is Excellent

Monday, November 5, 2007 

about_window.jpg

Not too long ago I got on a plane with Thunderbird, having transitioned to IMAP, woke my laptop in flight and found my imap mail cache had gotten borked. Five useful work hours wasted. So in my searches for “Thunderbird Disconnected Problems” I found mention of this program called “Mulberry” that didn’t have these problems. I had looked at Mulberry years ago and it was cool, but fee and Eudora was then current and free so I didn’t try it out. I am so glad I found it again. Mulberry handles disconnected IMAP perfectly, has a fast powerful search, and is well-organized. I’ve had no problems and I’m using it to write this now on an 11 hour flight.

Mail Compose Window.jpg

At the outset, it is clear this is the vision of a single programmer not the work of committee and as such it is quirky and has some unique solutions. I wouldn’t say it is more quirky than Eudora but at first one will definitely spend time searching for functions and consulting the somewhat thin documentation. The basics are easy enough, but some advanced features are non-obvious.

Further, Mulberry is Correct. That is it is a fairly precise implementation of just about every mail standard, including some that are still emerging. Not surprising as the author, Cyrus Daboo, has also written some of the key server-side programs that run the web, including some of the really hard bits like the SASL authentication engine I use on my server and one of the most popular IMAP servers. If something doesn’t connect it is because the other program (the server or whatnot) is making a mistake. This is great as far as it goes, but some non-RFC compliant usages have become commonplace and sticking to the RFC can cause problems. An example I found quickly was that the Message-ID: header Mulberry generates is constructed as unique-message-string@[client.dotted.quad] (something like 3499345954.0253243@[192.168.15.101]). This is correct, but the standard is to use @my.smtpserver.com, and using a non-fully qualified extension (the dotted quad, not a valid domain name). The dotted quad looks spammy to spam filters, and in particular when the client is on NATed DHCP, the private IP (192.168.etc) it looks bad. So Mulberry sourced mail might get a slightly higher SpamAssassin score (it is not a fatal test, but it can’t help) and my procmail spam filter looks for disagreement as a test so I can’t email myself notes to my own account – I have to send them to my MIT account.

Cyrus says he is going to fix this.

Which brings me to another wonderful feature of Mulberry: it has great support from the mailing list and author. You won’t go more than 24 hours without an answer to the most technical questions. And as it is in active development, any bugs are going to be fixed. Compare this to a MS product where that is not going to happen.

Mulberry’s mail interface took me a little getting used to. For example the mailbox list is organized a little differently and single clicks open new mailboxes in the next pane and the message in the pane below it, but this behavior can all be customized in the Window->Options… menu including, critically for me: do not mark previewed message as read.

Mail_window.jpg

Another good trick is automatically moving read messages out of the inbox. I haven’t been entirely satisfied with the sort options: the unread messages always seem to sort in the reverse order of what I want, putting the messages I need at the interface between the read and unread messages, rather than at the top or bottom. But the auto move mechanism works well for my inbox and lets me sort the inbox by date, it being all unread mail, the read mail automatically being moved to an archive.

I spent some time figuring out two wonderful features: Mulberry (along with GCalDaemon) supports off-line calendar sync with Google Calendar (YAY! I can answer email about my calendar while I’m on a plane and even schedule a meeting!) and I can sync to ScheduleWorld’s LDAP server (which syncs to my phone address book and my work Outlook address book). And since I use ScheduleWorld to sync my work Outlook calendar to Google calendar, I’ve got all my important information at hand, even in the air. I wrote up the steps to make these tricks work on the Mulberry Wiki.

calendar.jpg

Even the search function is fast – entirely tolerable though perhaps not quite real-time like Google Desktop, but then again you don’t need to open inane stupid brain dead IE to perform the search like Google Desktop forces you to.

Mulberry is great. It works really well, it is stable, it works offline (disconnected), it syncs right, it has a very good offline calendar client, IMAP support seems flawless, it has great keyboard shortcuts, and fast advanced search. It does everything I need and it is now free, open source, and available for Windows, Linux, and Mac OSX.

Posted at 00:00:20 GMT-0700

Category: PositiveReviewsTechnology

Linux 342

Wednesday, August 15, 2007 

An IBM 342 with a ServeRAID 4lx is a fine machine, but getting Linux to install is less the effortless. Emacs!

2d238159.jpg

I’m trying to get zoneminder to work on this very nice IBM 342 with a serve raid card and some good drives and 3 video capture cards. The thing should be able to capture 12 streams of video simultaneously, or 6 at full 30FPS. But getting Linux variants to properly recognize the serveraid card is a challenge.

The Mandrake LiveCD install works great on IDE systems, no problem at all. But it doesn’t see the serveraid, so that one was out. Gentoo saw the serveraid card, and since video capture and real time analysis is one of those things that would be good to do fast, the gentoo optimization scheme seemed promising, but it wasn’t. Just a miserable series of failed compiles and fixes that went on endlessly.

So from there to Debian, which is very nice and since it is the parent of Ubuntu and there’s an Ubuntu package and Carolyn loves Ubuntu, that seemed worth a shot. It does see the Serveraid, but there seems to be a bug in the IPS.o driver which reared it’s irritating head during package installs causing hangs, even after I updated the firmware to 7.12.12.

So that was out. On to a distro officially supported by IBM: Suse. That installed great, easy no problem, detected all the ADCs on the capture cards and everything. Very easy to install, but there are some weird bugs with ffmpeg that hung the compile of Zoneminder. It descended into another endless series of patch and edit and retry effort to get through the compile….

Then I saw that Fedora 7 has an RPM in the main distro for ZoneMinder. It is officially supported by IBM and seems rock solid. So far the network install has gone well – the install CD is only 7.71 MB (!) and it seems tentatively promising… it’s on the “Starting install process” screen, which is supposed to take several minutes. As it may need a few GB of data, I’ll give it some time. Unfortunately Fedora doesn’t support CD installs and the 342 has a laptop style CD-ROM drive, so doing a DVD install is out of the question. Network installs are efficient if you only have to do them once, but the retry is all penalty download.

Posted at 15:05:15 GMT-0700

Category: Linux