Posts Tagged computing

Index a DataFrame subset on string column name in Julia

julia> using RDatasets

julia> iris = dataset("datasets", "iris")

julia> iris[iris[:Species] .== "setosa", :]
50x5 DataFrame
| Row # | SepalLength | SepalWidth | PetalLength | PetalWidth | Species |
| 1 | 5.1 | 3.5 | 1.4 | 0.2 | "setosa" |
| 2 | 4.9 | 3.0 | 1.4 | 0.2 | "setosa" |
| 3 | 4.7 | 3.2 | 1.3 | 0.2 | "setosa" |
| 4 | 4.6 | 3.1 | 1.5 | 0.2 | "setosa" |
| 5 | 5.0 | 3.6 | 1.4 | 0.2 | "setosa" |
| 6 | 5.4 | 3.9 | 1.7 | 0.4 | "setosa" |
| 7 | 4.6 | 3.4 | 1.4 | 0.3 | "setosa" |
| 8 | 5.0 | 3.4 | 1.5 | 0.2 | "setosa" |
| 9 | 4.4 | 2.9 | 1.4 | 0.2 | "setosa" |

Tags: , ,

UTF-8 and Unicode FAQ

From Evernote Note


Multi-Hop SSH Tunnel from one host via another

It’s simple in its basic form!

$ ssh -L <local_port>:<host2>:<host2_port> -N <username>@<host1>

This sets up an SSH tunnel from the machine you are at to HOST 2 via HOST 1.

Tags: , , , ,

Advanced Perl debugging with Aspect

So I had this huge bunch of Spreadsheet::WriteExcel code where I was generating Excel sheets from some statistics I was gathering and I noticed that everytime I opened the Excel file I got a message with something along the lines “File error: data may have been lost”, but on a casual look the spreadsheet looked fine… but of course the devil is in the details. It turns out that if you happen to Spreadsheet::WriteExcel::write in the same cell twice (or more I guess :) ) that error message is what my Excel produces…. So how to find where the problem is… in this huge bunch of writes…

Enter Adam Kennedy’s excellent Aspect library for Perl, a truly brilliant module!! Thanks Adam!

How did I use it in this case? Here goes…

.... lots of other code....

use Aspect;

my $pointcut = call qr/Spreadsheet::WriteExcel::Worksheet::write$/;
# Observe the $ at the end, otherwise write_string will also match, and we don't want that

...... code passes ...

sub in_my_big_excel_write_block {

my %spcoords = ();
my $before_write = before {
my @args = @{$_->params};
if ( $spcoords{$args[1]}{$args[2]} ) {
croak "Will do double write at coord: ["
. $args[1] . "," . $args[2] . "] = >" . $args[3] . "<\n" . "Previous write at coord: [" . $args[1] . "," . $args[2] . "] = >" .
$spcoords{$args[1]}{$args[2]} . "<\n"; } $spcoords{$args[1]}{$args[2]} = $args[3]; } $pointcut; .... lots of $worksheet->write(...) code....


Now my poor perl script will die with a message telling me where and what I tried to write double and what I wrote there previously, now it’s easy to find!


A more general debugging tip would be this simple “before” advice:

before {
if( ($cnt % 1000) == 0 ) {
my @args = @{$_->params};
print "Calling " . $_->sub_name . " with args : " . Dumper(@args) . "\n";
sleep 1;
} $pointcut;

Neat huh!? And trust me, this is only some small simple example of the power of the Aspect library.

Tags: , , ,

Backing up Windows to Readynas with Cwrsync, an rsync package for Windows

I finally figured out a nice backup solution for our Windows laptop.

Some background, the machine is not constantly on so I wanted to have the backup initiated from the laptop side when it was up and running, and since it is a quite old laptop with a slow wireless connection I further wanted to use rsync to transfer minimal amount of data. In addition to this I preferred having the backup in a plain file structure rather some obscure windows backup format AND I wanted the backup side to hold more data than was possible on the laptop so we can delete some data on the laptop when the disk keeps filling up but I didn’t want this removed on the backup side.

So to facilitate all this we bought a Netgear Readynas NV+ with 2×1 TB mirrored disks. On this excellent machine I enabled rsync and activated rsync on the “backup” share.

Then, to the windows machine, I downloaded cwrsync an excellent stand alone packaging of rsync for windows (you don’t need to download Cygwin to use it). You can find cwrsync here.
Cwrsync supports utf8 over rsync (in contrast to DeltaCopy), something that is essential if you live in Sweden and use å ä ö in filenames.

I then created this:

cd "C:\Documents and Settings\USERNAME\Mina dokument"
rsync -avz "." " dokument"

small BAT script (based on the example included in the Cwrsync package, which sets up path’s to cwrsync etc. ) on the windows machine which recursively backs up all new or changed files in “Mina dokument” to the Readynas machine.

This small script was put on the Desktop (Skrivbordet in swedish), now all my better half has to do is to double click the BAT file and the backup is magically done in the most efficient way.

I’ll probably add a backup job on the Readynas to backup THAT backup to another NAS too… Did I mention that we DO NOT want to loose our pictures and videos of our daughter.

Tags: ,

Apple Time Capsule update to 7.5.2 firmware

Just a small note on this update, the 7.4.2 update was really bad for me (one Macbook Pro and one Windows XP machine), but the 7.5.2 seems to be quite stable for both the Macbook and the Windows XP machine.

Update 2011-01-23: The 7.5.2 is actually very stable and I can really recommend it. The wireless connections both to the Macbook and the Windows machine is much more stable!

Update 2011-03-06: Well… nope, they have fucked up agan! :P :P The wireless connection IS more stable than before but it seems that under heavy load (just a theory at the moment) it hangs and then IT IS IMPOSSIBLE TO restart with the LAN cable plugged in.

Update 2011-03-13: Ok, after some experimenting and a suggestion from my brother in law, it seems that setting the wireless channel to a fixed value (1 in my case) seems to have stabilized things.


Mac OS X Terminal och åäö

Har inte upplevt några problem förut, men nu har det börjat strula…

ÅÄÖ i Mac OS X Terminalen har blivit ett elände. :(

Det är 3 olika problem.

1. Få ut åäö från applikationer, t.ex ‘cat textfil.txt’ (där filen innehåller åäö) eller från t.ex Perl program
2. Skriva åäö på kommandoraden, för mig har den nu börjat ge ingenting (ok, ett “beep”) eller “(arg: 6)”
3. Få ‘ls’ att korrekt lista filer med åäö i filnamen.

Problem 1:
Löses genom att ändra terminal settings till Western (ISO Latin 1) Se bild.

Terminal Settings

Mac OS X Terminal Settings

Problem 2:
Löses med en kombination av lösningen till problem 1 och detta tips på
Dvs, skapa en fil i hemkatalogen som heter “.inputrc” och innehåller följande:

set convert-meta off
set input-meta on
set output-meta on

Om man bara skapar .inputrc så blir det knas när man t.ex skriver åäö, följt av 3 backspace som då backar tillbaka in på prompten för mig. Och att bara skapa .inputrc löser inte Problem 1 (för mig iallafall).

Problem 3:
Har jag ingen lösning på. :(
När jag gör ‘ls’ i en mapp med filer men åäö i filnamnen så visas “a?” istället för å, “a??” isf ä och “o??” isf ö och jag kan inte tab-completa filnamnet, dock så kan man skriva in filnamnet som det står eller copy-pasta det. Om jag gör export LC_ALL=”sv_SE” i terminalen och har “Terminal Settings” satt till UTF-8 då ger ‘ls’ svenska tecken i llistningen men då är jag tillbaka med Problem 1. :( Har jag Terminal Settings satt till Western (ISO Latin 1) så blir åäö några andra konstiga tecken (typ Ì).

Tags: ,

Downgrade 1T Apple Time Capsule from 7.4.2 to 7.3.2 firmware

Well, after upgrading to 7.4.2 firmware of my 1T Time Capsule and realized it sucks (our Windows XP machine gets dropped connections all the time, the MacBook stays up a bit better but very frequent drops there too), I was looking for the old 7.3.2 firmware, but Apple in their infinite wisdom has removed the possibility to download this firmware form their 7.3.2 download page, I finally found this link:

on this:

page, and concluded that it works for the Time Capsule also even though it is mainly the AirPort Extreme 802.11n Base Station that is discussed on the page.

If they end up removing that one to, feel free to contact me (by leaving a comment here) to get hold of the 7.3.2.basebinary.

Tags: ,

iTunes db on NAS – no end of pain!

So, being a very happy Apple customer it saddens me to no end, the problems I have with iTunes and having its database on my Apple Time Capsule (NAS).

Having a Macbook Pro I’m sorta limited on disk space so I don’t wanna have all my music and podcasts and iTunes University stuff (there’s some GREAT stuff on iTunes University, check it out, and it’s free!) on my Macbook disk so I thought “Well, why not just put it all on my Time Capsule, lots of space there, all Apple stuff too so it should work fine!”…. and there it began, my “no end of pain”.

I did the transfer of the data completely according to the book and I have everything the way Apple wants it (synch db, copy imports, manage names, etc) but the bastard just keeps mucking up the whole frikking time!!! :(

It looses the network connection for a moment and then suddenly iTunes decides that my whole library is GONE, “Wooops, better silently change the iTunes directory to the default Music folder”…. Unaware that iTunes made this clever move, I innocently want to sync my iPhone, because that is what we responsible users do, iTunes then goes… “Wow, he seems to have empty his whole library, and since we are sync’ed I’d better wipe his whole iPhone without asking first”…. NO, NO, NO YOU BASTARD!!!

So painstakingly I put it all back, which takes hours, but then all my data that I have put in my Apps are…wait for it… wait for it…. GONE!!!! AAAARRRGGGHHH!!! THIS IS NOT HOW IT IS SUPPOSED TO BE!!!

Not only is there the above problem, additionally it starts adding any stuff to this “new” iTunes folder which it silently changed to which I have to fiddle over to the correct place later on when I realize what has happened. And every time I change back it has to “Check my iTunes library” which takes forever, THEN it asks me to “Update all the frikking songnames and whatever frikkin else” so it loops over all my stuff and “changes” it to the correct names, which I have already told it to do 70 times, so they bloody well should be correct by now… SIGH!

The best way I have found of preventing this STUPID silent moving of the iTunes folder is to create a write protected link in the Music folder to the correct place on my Time Capsule, then at least I find out about it and can prevent it from happening, then it usually reports something along the lines that my iTunes folder is write protected and gives up… HA!!! GOT YA!!! :P

My latest hoe is that it hangs when i sync my iPhone, it does all its stuff and then when you figure it is finished, it just hangs on “Syncing iPhone”… buhuuuu!

Apple! Fix! Now! This is not funny anymore, it wasn’t even funny from the start and it is beneath you!!

Tags: , , ,

A real Turing Machine

Isn’t this too cool!!??

I want one!

Tags: , , , ,