perl

28C3 Scariest Talk of the Day

Wednesday, December 28, 2011 

We attended Effective Denial of Service attacks against web application platforms by Alexander “alech” Klink and Julian | zeri where they described a really, really easy to implement denial of service attack that exploits an artifact of hash checking which is computationally intensive when the hash table is filled with hash collisions. It is fairly easy to find 2-4 character hash collisions for a given hash functions (and there are only a few variations in use) and as hash operations are performed by default on all POST and POST-like functions, which take (by default) from 2-8MB of data, one can easily tie up a computers CPU effectively indefinitely.

The researchers tested the attack on most web languages in use (and all in common use – only Perl is deployed safe (since 2003) and Ruby 1.9 has a patch available. Every other OS is vulnerable. Today. The attack is only a POST option with a table of delimited hash collision values. You could copypasta a working exploit, it is that easy. The vast (vaaast) majority of sites on the web run PHP, and 1 Gbps of attack vector bandwidth could take down 10,000 cores. With ASP.NET, that 1 Gbps can hold down 30,000 cores cRuby 1.8 (not patched, about half of Ruby installs): that 1 Gbps can keep a million cores tied up.

Yow.

Posted at 18:32:59 GMT-0700

Category: EventsTechnologyTravel

Cleaning Out Duplicate IMAP messages

Saturday, October 17, 2009 

Find some great IMAP scripts here.

There are a number of ways to end up with a lot of duplicate messages in an IMAP folder, and while IMAP tends to handle very large stores gracefully, it is possible to hose things.  On my 32 bit server and with Mulberry as a client things get weird after about 15,000 messages in a single folder.

Google does some odd things and at one point a periodic check of my gMail account resulted in about 70,000 messages in a single folder, which definitely caused some chaos.

I thought that was pretty impressive, but my girlfriend just managed to get 144,000 messages in a single folder.  Woo Hoo!!!  High Score.

Anyway, things like the dedup plugins for Thunderbird can just make things worse at that point as they seem to fail gracelessly on very large message counts.

I found that Rick Sander’s perl scripts are the best way out of this difficult situation.  delIMAPdups.pl solves the problem without running out of memory or munging files.  I haven’t had any lost data and just tested by clearing about 1400 dups out of a directory of 15,000 messages (my 2009 store to date).
/.delIMAPdups.pl -S example.com:993/user/pass -m INBOX.2009 -p
-m is the mailbox to expunge
-p is purge
-S means use SSL

Posted at 23:49:53 GMT-0700

Category: FreeBSDTechnology

I hate thunderbird

Monday, October 8, 2007 

So once, long ago, I moved to IMAP on my server. I wanted to move there with my trusty Eudora client that I’ve been using since about 1993. Sure, I flirted with other mail systems, but they screwed me and what I care about most was:

  1. Never Lose Data (early versions of Thunderbird were not so good about this for me)
  2. Search my several gigabyte database of mail fast enough to be useful.

Now outlook is absolutely intolerable about this last point. Search in all Microsoft products, indexed or not, is so painfully slow one might as well go on vacation. It is incomprehensible to me how it can suck so bad. I remember in 1990 using OnLocation and searching my entire computer (all 20MB of it) in a fraction of a second. Sure it was less data, but it was also doing it on a 33Mhz 68030.

Eudora lets me find my mail. Eudora lets me get my job done. Unfortunately Eudora can’t search an IMAP mailbox unless it is connected. WTF?

OK, time for Thunderbird. But Thunderbird is so not ready for prime time. There’s the massive delays to open any of my larger mailboxes, even to show titles (14,000 messages in a mailbox is NOT too many, who uses this? Kids?) Second it gets confused easily communicating with the IMAP server which tends to lock it up indefinitely. Still, it does cache locally and the built-in search, while interminably slow is faster than Microsoft Search (but doesn’t search across accounts! Hello!). I’m hoping Google Desktop Search will help. Initial results are promising. And Penelope could be very cool. Especially if they add indexed search.

One little change I had to make for Thunderbird was given at this fine site:

perl -p -i -e 's/^MAXDAEMONS=40/MAXDAEMONS=80/g' /usr/local/etc/courier-imap/imapd

perl -p -i -e 's/^MAXPERIP=4/MAXPERIP=40/g'  /usr/local/etc/courier-imap/imapd

Update - 9/9/2010

I’m now using Thunderbird (3.1).  It hasn’t lost data yet.  It has a nice fast search.  I still don’t like it as much as Mulberry for basic mail functions, but it shows pictures in line and I can quickly toggle between HTML mail (to insert inline pictures and screen grabs) and text mail.

These are useful basic features and I wish Mulberry had them.

Posted at 15:45:05 GMT-0700

Category: FreeBSDReviewsTechnology

ZoneMinder on FC7

Saturday, August 25, 2007 

26828649.jpg

Overview
Zone Minder Config ZoneMinder 1.22.3 on Fedora Core 7

There are useful instructions at this URL

Do a basic install of FC7.

  • KDE seems to work better than gnome.
  • Remove unnecessary SW to speed install (desktop stuff)
  • Add Server and Development to get the right tools
  • Add https://www.systutorials.com/additional-repositories-for-fedora-linux/ as an RPM source
  • Make sure the necessary holes are in the firewall at 80

Add necessary bits
yum install mysql-server mysql-devel php-mysql pcre-devel \
perl-DateManip perl-libwww-perl perl-Device-SerialPort \
perl-MIME-Lite perl-Archive-Zip

updating perl (some will be installed already)
perl -MCPAN -e shell
install Bundle::CPAN
reload CPAN
install Archive::Tar
install Archive::Zip
install MIME::Lite
install MIME::Tools
install DateTime
install Date::Manip
install Bundle::libnet
install Device::SerialPort
install Astro::SunTime
install X10
quit

FFMPEG install

Note that getting the FFMPEG libraries installed so they work is a nightmare. I followed these instructions and they seemed to work:

First add the x264 libraries and devel from livna via software manager

If the database hangs try
rm /var/lib/rpm/__db*
rpm --rebuilddb
yum clean all

svn checkout svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
cd ffmpeg/

./configure --enable-shared --enable-pp \
--enable-libx264 --cpu=pentium3 --enable-gpl

make
make install
nano /etc/ld.so.conf

add the line “/usr/local/lib”
ldconfig

System demons

chkconfig --add mysqld
chkconfig --level 345 mysqld on
chkconfig --level 345 httpd on
service mysqld start
service httpd start

add to /etc/sysctl.conf to increase shared memory limit
kernel.shmall = 134217728
kernel.shmmax = 134217728

Zoneminder Install

Check the latest version of zoneminder at https://web.archive.org/web/20110310142847/http://www.zoneminder.com:80/downloads.html

wget https://zoneminder.com/downloads/
tar xvfz ZoneMinder-1.22.3.tar.gz
cd ZoneMinder-1.22.3

patch it

https://web.archive.org/web/20110501001851/http://www.zoneminder.com:80/wiki/index.php/1.22.3_Patches

The configure command I used is:
./configure --with-webdir=/var/www/html/zm \
--with-cgidir=/var/www/cgi-bin ZM_DB_HOST=localhost\
ZM_DB_NAME=zm ZM_DB_USER=zmuser ZM_DB_PASS=zmpass \
CFLAGS="-g -O3 -march=pentium3" CXXFLAGS="-g -O3 \
-march=pentium3" --with-ffmpeg=/usr/bin \
--with-webuser=apache --with-webgroup=apache

putting a reasonable user name for “zmuser” and password for “zmpass”

make
make install

If make barfs with
/usr/local/src/ZoneMinder-1.22.3/src/zm_mpeg.cpp:284: undefined reference to `av_free(void*)'
try
”in src/zm_mpeg.h starting on line 26, add the lines with the + (removing the + of course) The other lines are just for reference and should be already in the file.” from this reference (lost to the void, alas).

nano src/zm_mpeg.h

#define ZM_MPEG_H
+extern "C" {
+#define __STDC_CONSTANT_MACROS
#include <ffmpeg/avformat.h>
+}
#if FFMPEG_VERSION_INT == 0x000408

Install scripts
install scripts/zm /etc/init.d/
chkconfig --add zm

Create and configure the ZoneMinder database
mysql mysql < db/zm_create.sql
mysql mysql

at the mysql prompt:
grant select,insert,update,delete on zm.* to \
'zmuser'@localhost identified by 'zmpass';
quit

mysqladmin reload

GO!
service zm start

you should get a nice green [OK].

http://127.0.0.1/zm

Black Screen? Go Faster?
No php?
If you have issues make sure you have installed apache php and perl modules.

IJG SIMD jpeg should double performance.
http://cetus.sakura.ne.jp/softlab/jpeg-x86simd/jpegsimd.html#source
* requires nasm which wasn’t installed. Use package manager.
wget http://cetus.sakura.ne.jp/softlab/jpeg-x86simd/sources/jpegsrc-6b-x86simd-1.02.tar.gz
tar xvfz jpegsrc-6b-x86simd-1.02.tar.gz
cd jpeg-6bx
./configure --enable-shared --enable-static
nano Makefile

* Change the CFLAGS from O2 to O3 and add
-funroll-loops -march=pentium3 -fomit-frame-pointer

make
make test
make install

identify the libraries to the system
ldconfig

I also copied the installed files from /usr/local/bin to /usr/bin:
cp /usr/local/bin/cjpeg /usr/bin/cjpeg
cp /usr/local/bin/cjpeg /usr/bin/cjpeg
cp /usr/local/bin/cjpeg /usr/bin/cjpeg
cp /usr/local/bin/cjpeg /usr/bin/cjpeg
cp /usr/local/bin/cjpeg /usr/bin/cjpeg

/etc/init.d/zm restart

NetPBM resizes the JPEGS and faster is better: compile and install
cd /usr/src
svn checkout https://netpbm.sourceforge.net/ netpbm
cd netpbm
/usr/src/netpbm/configure

Answer the questions (GNU and then defaults – I didn’t have TIFF or VGA libs, so “none”)
vi Makefile.config
I added -march=pentium3 to the CFLAGS at the end of the file
make
make package
/usr/src/netpbm/installnetpbm

accept defaults

cabozola install

* package add Ant (it needs ant, but it wasn’t installed by default)
cd /usr/src
wget https://web.archive.org/web/20220526174548/http://www.charliemouse.com/code/cambozola/cambozola-latest.tar.gz
tar xvfz cambozola-latest.tar.gz
cp /usr/src/cambozola-0.68/dist/cambozola.jar /var/www/html/zm
chmod 775 /var/www/html/zm/cambozola.jar

Posted at 01:44:34 GMT-0700

Category: LinuxTechnology