Posts Tagged linkedin

Index a DataFrame subset on string column name in Julia


julia> using RDatasets

julia> iris = dataset("datasets", "iris")

julia> iris[iris[:Species] .== "setosa", :]
50x5 DataFrame
|-------|-------------|------------|-------------|------------|----------|
| Row # | SepalLength | SepalWidth | PetalLength | PetalWidth | Species |
| 1 | 5.1 | 3.5 | 1.4 | 0.2 | "setosa" |
| 2 | 4.9 | 3.0 | 1.4 | 0.2 | "setosa" |
| 3 | 4.7 | 3.2 | 1.3 | 0.2 | "setosa" |
| 4 | 4.6 | 3.1 | 1.5 | 0.2 | "setosa" |
| 5 | 5.0 | 3.6 | 1.4 | 0.2 | "setosa" |
| 6 | 5.4 | 3.9 | 1.7 | 0.4 | "setosa" |
| 7 | 4.6 | 3.4 | 1.4 | 0.3 | "setosa" |
| 8 | 5.0 | 3.4 | 1.5 | 0.2 | "setosa" |
| 9 | 4.4 | 2.9 | 1.4 | 0.2 | "setosa" |

Tags: , ,

Advanced Perl debugging with Aspect

So I had this huge bunch of Spreadsheet::WriteExcel code where I was generating Excel sheets from some statistics I was gathering and I noticed that everytime I opened the Excel file I got a message with something along the lines “File error: data may have been lost”, but on a casual look the spreadsheet looked fine… but of course the devil is in the details. It turns out that if you happen to Spreadsheet::WriteExcel::write in the same cell twice (or more I guess 🙂 ) that error message is what my Excel produces…. So how to find where the problem is… in this huge bunch of writes…

Enter Adam Kennedy’s excellent Aspect library for Perl, a truly brilliant module!! Thanks Adam!

How did I use it in this case? Here goes…

.... lots of other code....

use Aspect;

my $pointcut = call qr/Spreadsheet::WriteExcel::Worksheet::write$/;
# Observe the $ at the end, otherwise write_string will also match, and we don't want that

...... code passes ...

sub in_my_big_excel_write_block {

my %spcoords = ();
my $before_write = before {
my @args = @{$_->params};
if ( $spcoords{$args[1]}{$args[2]} ) {
croak "Will do double write at coord: ["
. $args[1] . "," . $args[2] . "] = >" . $args[3] . "<\n" . "Previous write at coord: [" . $args[1] . "," . $args[2] . "] = >" .
$spcoords{$args[1]}{$args[2]} . "<\n"; } $spcoords{$args[1]}{$args[2]} = $args[3]; } $pointcut; .... lots of $worksheet->write(...) code....

}

Now my poor perl script will die with a message telling me where and what I tried to write double and what I wrote there previously, now it’s easy to find!

🙂

A more general debugging tip would be this simple “before” advice:


before {
if( ($cnt % 1000) == 0 ) {
my @args = @{$_->params};
print "Calling " . $_->sub_name . " with args : " . Dumper(@args) . "\n";
sleep 1;
}
$cnt++;
} $pointcut;

Neat huh!? And trust me, this is only some small simple example of the power of the Aspect library.

Tags: , , ,

iPhone 3GS Personal Hotspot to iPad 2

Just wanted to confirm that Personal Hotspot does work on the iPhone 3GS and it is indeed possible to pair it with my iPad 2 WiFi only!! Yay!!

It requires iOS4.3.

One “funny” thing is that I had updated my iPhone 3GS to 4.3 but there where no “Personal Hotspot” available in the Settings. It turns out that I had to reset my network settings, AND to use the “correct” reset button.

For me it did NOT help by using the “Reset” under Settings->General->Network>Cellular Data Network->Reset Settings”.

But when I used the General->Reset->Reset Network Settings, the iPhone 3GS rebooted and at restart the Personal Hotspot was indeed available under Settings->General->Network !

On the iPhone 3GS the Personal Hotspot uses Bluetooth and does not use WiFi sharing as in the iPhone 4. So you will have to activate Bluetooth on both devices and pair them to get it to work.

Tags:

Spela PS2 SingStar spel på PS3

Tänkte bara bekräfta att det gÃ¥r att spela SingStar spel som är gjorda till PS2’an pÃ¥ PS3’an. Jag tycker inte att det varit särskilt klart att det faktiskt gÃ¥r, men ikväll har jag lyckats testa det och det funkar!! Jag provade ikväll (äntligen!!) SingStar Sjung Med Disney (som bara finns till PS2) pÃ¥ vÃ¥r PS3.

Så hur går man då till väga?

Man mÃ¥ste ha en PS3 SingStar skiva till att börja med, man startar med den skivan i PS3’an och startar igÃ¥ng spelet, sen navigerar man till “Välja LÃ¥t Menyn” där trycker man pÃ¥ “Select” knappen pÃ¥ handkontrollen, dÃ¥ fÃ¥r man möjlighet att byta skiva och dÃ¥ kan man sätta in en Singstar PS2 skiva.

Observera att det gÃ¥r inte bara att starta igÃ¥ng spelet och trycka ut skivan för dÃ¥ stannar spelet, man mÃ¥ste gÃ¥ till “Välja LÃ¥t Menyn” och trycka “Select” pÃ¥ handkontrollen.

Tags:

ClickToFlash!

Beeing really sick and tired of Adobe Flash and how it hogs up the entire CPU and slows Safari down to a grinding halt I finally had enough and tried to find some remedy and found the EXCELLENT ClickToFlash. ClickToFlash is a Flash blocking plug-in for Safari on Mac OS X and there is just one thing to say about it, “Get it”!! (ok, two things…).

Excellent unobtrusive tool that simply just does the right thing!

Haven’t found any problems with it yet (as you can see WordPress works fine with it installed, including image uploads), but if I do, I’ll report it here.

[UPDATE] Having now run ClickToFlash for a couple of days I must say it works like a charm, love it!!! It’s like having a new computer, no more Mr Sirup computer!! Thanks to all the people who are developing ClickToFlash!!!

Tags:

Backing up Windows to Readynas with Cwrsync, an rsync package for Windows

I finally figured out a nice backup solution for our Windows laptop.

Some background, the machine is not constantly on so I wanted to have the backup initiated from the laptop side when it was up and running, and since it is a quite old laptop with a slow wireless connection I further wanted to use rsync to transfer minimal amount of data. In addition to this I preferred having the backup in a plain file structure rather some obscure windows backup format AND I wanted the backup side to hold more data than was possible on the laptop so we can delete some data on the laptop when the disk keeps filling up but I didn’t want this removed on the backup side.

So to facilitate all this we bought a Netgear Readynas NV+ with 2×1 TB mirrored disks. On this excellent machine I enabled rsync and activated rsync on the “backup” share.

Then, to the windows machine, I downloaded cwrsync an excellent stand alone packaging of rsync for windows (you don’t need to download Cygwin to use it). You can find cwrsync here.
Cwrsync supports utf8 over rsync (in contrast to DeltaCopy), something that is essential if you live in Sweden and use å ä ö in filenames.

I then created this:


cd "C:\Documents and Settings\USERNAME\Mina dokument"
rsync -avz "." "10.0.1.16::backup/WindowsBackup/Mina dokument"

small BAT script (based on the example included in the Cwrsync package, which sets up path’s to cwrsync etc. ) on the windows machine which recursively backs up all new or changed files in “Mina dokument” to the Readynas machine.

This small script was put on the Desktop (Skrivbordet in swedish), now all my better half has to do is to double click the BAT file and the backup is magically done in the most efficient way.

I’ll probably add a backup job on the Readynas to backup THAT backup to another NAS too… Did I mention that we DO NOT want to loose our pictures and videos of our daughter.

Tags: ,

A real Turing Machine

Isn’t this too cool!!??

I want one!

Tags: , , , ,

My bout with iPhone OS 4 and downgrade back to 3.1.3

So, curious on the iPhone OS 4.0 I wanted to testrun it on my iPhone 3GS, downloaded the upgrade from Apple, “option clicked” restore and browsed to the “iPhone2,1_4.0_8A230m_Restore.ipsw” file and off we go…

Well I don’t know about this beta, it wasn’t very stable for sure, the worst problem being not able to read mail properly. Also all my contacts disappeared when I upgraded, not very impressive (my songs where still there though curiously enough). Weren’t too impressed with the multitasking, I ended up seeing “warning low memory” a couple of times and having to go in to the “task manager” and kill some apps, hopefully this is some we won’t see in the production version because then Apple is seriously going in the wrong direction!

Another thing that bothered me a bit, I have very frequently used the “home button double click” mapped to the iPod and that will likely disappear now with the “task manager” mapped to “home button double click”. Too bad, fortunately it seems to still work when the screen is locked…

Nevertheless, soon got tired of the bugginess and wanted to downgrade back down to 3.1.3… not soo easy it turns out…

First tried the normal “Restore procedure”, no luck, just got something along the lines of “Cannot download software” (no error code) and then it just referred me to this Apple page.

So then I tried the “option click” restore and choose the “iPhone2,1_3.1.3_7E18_Restore.ipsw” file, iTunes went off and fiddled around with the iPhone for a while, but ended up with Error code 1015.

I fiddled around with various variants of this for a while trying both the 3.1.3 version and the 4.0 version but no luck.

What’s worse I had obviously ended up in what is called the “Apple Logo or Recovery Mode loop”, i.e it is now impossible to boot the phone it just keeps coming back to recovery mode and when plugged into iTunes, it will just ask you to restore and we have already concluded that didn’t work for me, so what now??

Found this and this page and combining the two did the magic. Like it says on the first page, it can take several tries before it works, for me it took two times before I finally managed to get it back to 3.1.3.

Tags: , ,

Recover crashed Windows machine

So, the inevitable finally happened to our old Windows HP Laptop, it crashed… We have been waiting for it to happen, it is, after all OLD, but I must say it has done a really good job hanging in there this long.

Yep, we have been making backups, but not as regularly as with the Mac and the excellent Time Machine of the simple reason that it is more of a hassle doing it under windows.

So, although most files where backuped there where still some missing. So what to do? The machine didn’t even boot so no way of getting in and copy files to a USB stick. Considered trying a repair with the Windows XP CD, but I don’t really trust them not to do stupid things so discounted that option.

Having done this stuff before I remember beeing really happy with the System Rescue CD (SRCD), I consider it to be, by far, the best in the category! This is one excellent package! Its a Linux Live CD with all the stuff you’ll ever need, including a really good website. So downloaded that and burnt it to a CD and stuffed it into the old HP.

Linux booted with no problem, next step is to find the hard drive:
fsarchiver probe simple
showed it to be: /dev/hda1 with an NTFS filesystem, this used to be an issue, but not anymore. So mounted the hard drive with: ntfs-3g -o ro /dev/hda1 /mnt/windows The /mnt/windows exists from the start so no need to create that directory, the “-o ro” option mounts it read only so we can’t misstakenly remove or change anything on the drive, that feels good in these situations.

Ok, so the machine is up and running and we have access to all the files on it, good. My plan now was to backup the files and then try to reinstall Windows. So how to backup the files? Initially I figured I’d copy them over the network using sftp which was the suggested approach on the System Rescue CD website. So I got the network up and running by connecting a network cable and issuing dhclient eth0. No problem. ifconfig showed that the network was up and running and the ip-address I got. OpenSSH is already up and running on the crashed machine so I started cyberduck on my Mac and used sftp to connect to the crashed machine.

I then started copying the files I wanted, but unfortunately it would take about 13 hours. I didn’t really file like waiting that long, so in parallel I started looking for a USB drive. I found an old one with 220 GB free and connected that to the crashed machine. USB is of course supported by SRCD, so again, I ran fsarchiver probe simple to find the USB drive, it showed up as /dev/sda1 with a vfat filesystem on it.

So mounted that one with mount -t vfat /dev/sda1 /mnt/backup again the /mnt/backup already existed so no need to create that either.

Now I got a small issue, since normal users don’t really know (and shouldn’t need to know either) alot about filesystems they are blissfully unaware of the issues that can occur when you create files with the swedish characters “Ã¥”, “ä”, “ö” and the like in the filenames so they tend to do just that. Now when I tried to copy files and directories with the above mentioned swedish characters (“åäö”) in them I got some error along the lines of “Could not create file, Illegal argument ” and I suspected that the “åäö”‘s were the issue. It turns out that I needed to unmount the “/mnt/backup” and mount it with mount -t vfat -o iocharset=utf8 /dev/sda1 /mnt/backup instead, then it worked fine, I’m not sure why this was since it says that “iso8859-1” is the default which certainly is capable of representing “åäö”, but maybe it was due to how the USB drive had beed formatted or something.

Ok, so now we are capable of handling “åäö”, so it is time to start copying the files. Next question, what to use to copy the files…? This might sound like a silly question but anyone that has tried to copy large amount of files with windows explorer knows what kinds of problems you can have. For example a file somewhere in the middle of the copy process is broken or some other error occurs and the file copying stops dead in its track somewhere in the middle of everything and you have no idea what is copied or not and you basically have to try to find the offending file, remove it and start over, and again… and again… 😛

Now, there are a couple of options on Linux:

cp – The ordinary copy program
dd –
tar-
dar-
rsync- A very competent file sync program from the Samba team

I ended up using rsync mainly since “cp” doesn’t show any progress, tar, dar creates an archive which I didn’t really want, other than that I have very high confidence that rsync will “do the right thing”.

To use rsync to just do a simple copy and show progress I use the following command: rsync -avh --progress /mnt/windows/Documents\ and\ Settings/Tess ./ (I had already cd’d to /mnt/backup/). As you can see I copied the “Documents and Settings/” folder, but I don’t really know if it contains everything that is needed, I hope that most applications save their data in the “Application Data” folder under that directory. Of specific interest was Firefox Bookmarks and Thunderbird emails and adressbook.

When all files where copied with rsync (40 GB) the sftp had managed to copy around 17% (6GB). Ok, so what’s next?

If I need to reinstall Windows, “My Documents” might be whiped, don’t want that, so while still running SRCD I moved everything in the “Documents and Settings/<username>/Mina Document/” folder to “c:/bak”, which is /mnt/windows/bak on the SRCD (the “bak” folder was not there so I created that first with mkdir /mnt/windows/bak.

Ok, after having secured the files I feel more confident starting to experiment with the machine. I then booted from a Windows XP Professional CD and chose the “Repair using the Recovery Console”. Then I tested the boot configuration with the command bootcfg /list it turns out that it is broken. “Dir” did not work either probably because the “Recovery Console” didn’t manage to find any “Windows Installation” to “log in” to. None of the “/rebuild”, “/scan”, “/add” options worked, so at this stage I wanted to delete the boot.ini file. I first had to make it deletable at all using the commands:

attrib -H C:\boot.ini
attrib -R C:\boot.ini
attrib -S C:\boot.ini

Now was able to delete the boot.ini, my first and so far only mistake, I should of course just have moved it to some other filename.

At this point I tried to boot the system in safe mode. And this time it actually managed to boot it using “Safe mode”, the first time since the crash. After rebooting it can now boot normally even though it prints a message that it cant find a boot and informs that it uses “c:\..something..windows..something”, pretty remarkable…

The final step was using the Windows XP (pro) CD to create a new boot.ini file. After this and a reboot it was again back to normal… let’s see how long it will make it!

Tags: , , , , , , ,

Pipe Emacs buffer through Perl

Ok, this is sooo cool! 🙂 For us who love Perl and often have the feeling of “If I could just filter this emacs buffer through Perl, how happy I would be!” here’s a tip:

Step 1: Mark the buffer (or the parts of it that you are interested on running through Perl).
Step 2: Type M+| (“Escape” + “|” (vertical bar) on your normal keyboard) or
Step 2 (alt): Issue the “shell-command-on-region” elisp function
Step 3: Enter your perl magic, for instance ‘perl -ne “print if /gnu/”‘ to remove all lines which does not match “gnu”
Step 4: Be amazed!

Now, the result will end up in the minibuffer if it can fit on one line, otherwise a new “shell command output buffer” will be created and hold your stuff.

Tags: , , ,