Recently in Off Topic Tech Category

ptdell.jpgUniverse: I am resigning from Oracle.

I know the universe of interested parties shrinks every year as the sales of the WCI portal (née Plumtree) decline, Oracle promotes a different product, and old customers move on to new platforms. But! Some of you are still out there reading, and so thanks!

Fortunately for you all, I'm not going far. I'll continue working with the WCI portal for a long-time customer, Boeing, for whom I've consulted off and on, but mostly on, since 2004. So the blog entries will continue to sporadically pop into your RSS feeds.

I have three company laptops that I need to return. The newest one Oracle issued to me several months ago, and I'm sure it will be redeployed to another employee. The older ones, however, will likely be "decommissioned." Occasionally I read stories about crooks who buy old hard drives to recover their data and then engage in all sorts of nefarious crimes. I don't want my data open to that risk. Since I don't know exactly what Oracle's decommissioning process is, and since any company's processes may not be perfectly followed, I decided to take extra care to destroy the personal, customer, and corporate data that had been on the hard drives.

So here's what I'm doing tonight, and you probably should do something similar when you let go of your old laptops, whether you're disposing of an old personal machine or resigning from the job that had run its course:

  1. buddha-baby.jpgCopy any needed data off the old laptop (e.g. this photo from when kiddo was a newborn)
  2. Create a "live cd" or a bootable disk with a *nix operating system on it. I used Ubuntu (get it).
  3. Boot your old laptop from the CD. On my Dell laptop, I used F12 to get a one-time boot menu to select that I wanted to boot from CD rather than from the hard drive.
  4. Identify the partition name for your disk. I did this by going to System -> Administration -> GParted Partition Editor.
  5. Open a console.
  6. Type a command like this one at the prompt, where /dev/sda2 is my laptop partition to wipe:

    sudo shred -vfz -n 1 /dev/sda2

  7. Wait while the machine overwrites your entire disk first with random data, then with zeros.

That's it. There's not much left to find on the drive. This is a much better approach than just reformatting the drive, because reformatting merely clears the address tables for the disk but still leaves the data intact and retrievable by Dr. Evil who makes his business doing such things. Of course, you could be more fastidious than I was. Another blog gives a more detailed review of the technical issue and even more thorough ways to knock it out.

After erasing the data, I went the extra mile to installed Ubuntu. This way anyone who turns on the computer will be able to log in and see that nothing is readily available, and they'll also find it to be a generally useful machine.

Enjoy.

PS: Yes, I'm extraordinarily happy to move on from Oracle!

Love at First Boot: The D-Link DNS 323 NAS

| | Comments (1)
D-Link_DNS-323[1].jpg[Added Dec 29 2011] The best love endures overtime. My infatuation has passed. I cannot recommend this device for use outside the home. The default software only lets a user belong to a single group (e.g. finance) but not two (e.g. finance and managers). You can hack around this by using ffp then editing smb.conf, but you don't want to do that routinely. Also, ffp isn't compatible with the most recent firmware, and only the most recent firmware lets you do offsite backup with the CTERA plugin. Also, it requires SMB access requires old and insecure lanman auth, which presents problems when a hardened system like recent Ubuntu distributions want to connect (though you can rollback security on Ubuntu to connect). If I had it to do over again, I would buy the QNAP TS-239 Pro Turbo to get to know a more serious device with disk encryption, better backup options outside the device, better user/group management, etc. And now, the original post...




Remember that giddy feeling when in high school you first ate lunch on the grass with that special someone, the object of your springtime infatuation? Ahhh. So sweet. I'm reliving that feeling with my newly installed NAS. I tenderly call her "323" for short, but her parents call her "D-Link DNS-323 2-Bay Network Storage Enclosure." I can see beyond her toaster looks...

I don't blog about every early Christmas present, but this NAS is so geek-winningly hackable, and I wound up doing such a number on my home network for it that I can't help but share the story. This may be helpful to other web wanderers, just as I relied on many blog posts, discussion forums, and so forth to get set up.

Benefits

First the benefits of this relationship:

1. Network Attached Storage -- You know at the office how nice it is to always have access to those never-ending shared drives that corporate IT provides. I now have it at home. Instead of keeping only select music stored on my computer and the rest locked away on that external USB drive in my wife's office, it's all available. The old cables and plugs were a barrier to access.

2. Peace of Mind -- With RAID-1 and two SATA drives, my data won't get lost when a hard drive fails.  And every hard drive fails sooner or later.

3. Openness -- The 323 runs an embedded Linux, and D-Link built a hook to let folks access the core. Extend it with Subversion, SSH, MySQL, or if you're crazy enough you can even install a new Debian.

4. FTP -- The built-in FTP server and granular security model lets me access, share, or backup content from outside the home.

5. iTunes Server -- The device can discover its music then broadcast it to iTunes clients on the network.

6. Scheduled Downloads -- It can schedule downloads of files and folders from an FTP server, web server, or local network share. I don't want to fully rely on my web hosting providers to backup my data, and this lets me keep a copy too.

The feature list is rich, but not all of it applies to me -- yet. We'll see how my thinking shifts as she and I get to know each other better. Other people though are interested in its BitTorrent feature, UPnP AV server, or others

Now for some details.

The Hard Drives

Kermit[1].jpgIn keeping with the do-it-yourself offering, the 323 doesn't come with hard drives. It's just an enclosure. So what did I buy? I admit that I was driven by price rather than features, but I still wound up with a pair great drives. Amazon was selling Western Digital's energy efficient WD10EADS drives cheaper than any of the other 1TB options, at least with 7200 rpm. It's cool to be green (no matter what Kermit says), but I'm more excited about the cool temperature than the green energy savings. As drives heat up, the probability of failure increases dramatically. More on failures later. The 323 has a feature to monitor the temperature and at high levels, send an email alert and then shut down. I want this feature, but I also don't want it to ever be triggered. The drives were $69 each when I bought, but perhaps for the holidays they have since risen to $84.

Installing the drives to the 323 was easy. I just tore open the drive packages from Amazon, slid the front plate off the 323, and pushed in the drives. No tools required.

Improving the Home Network

In order for my wife and me to share the NAS, our laptops need IP addresses from the same network. Previously, we didn't have this. The Internet drop and primary wireless router (an old WRT54g) are in my office. Since her office is on the other side of the house, and since the house has built-in ethernet wiring from the location of the Internet drop, we put a secondary wireless router (an older BEFW11S4) near her office that pulls data from the ethernet port.That router though was configured the easy way, with DHCP enabled and placing her on a different network. I was on 10.1.10.x, and she was on 10.1.11.x. So here's what I did:

1. Made sure the primary router ran normally, with our ISP's Internet provided through the router's WAN port
2. Made sure the secondary router ran normally, with the primary router's Internet provided through the secondary router's WAN port
3. Changed the secondary router to use a static IP, which in retrospect may not have been necessary
4. Configured the secondary router's Setup->Advanced Routing page to both send and receive RIP 1, which may not have been necessary but one blog suggested
5. Moved the secondary router's ethernet cable from the WAN port to port 1 which does uplink.

That was it. Now when my wife connects to the secondary router by her office, it acts as a switch to get to the primary router, and the primary router gives her an IP address in 10.1.10.x so we can both communicate with the NAS.

Improving the 323

Out of the box the 323 is nice, but it really starts to get cool once you start treating it as a customizable Linux box instead of just a hard drive. The device has a thriving community supporting it, and it's a great example of how a company's decision to open their product up can improve its usefulness and cultivate buzz (e.g. this blog post). The best site for the product may be http://wiki.dns323.info/. I proceeded cautiously installing my first "fun_plug" file to execute my commands at startup based on instructions at that site. Once my feet were wet, I installed a package of Unix tools called "ffp" (Fonz fun_plug) following the instructions at nas-tweaks.net.

In no time at all, I had logged in through telnet, disabled that insecure service, set up SSH, and begun looking around. I then followed the instructions at another blog to install the usb-storage.ko module allowing me to mount Fat32 USB devices through the 323's USB port. I got my wife's old iPod Mini loaded up with little effort.

Breaking Up, not Backing Up, with Standard USB Hard Drives

After setting up a few directories on the hard drives with proper security, I powered up the old 250 gb USB drive that started all this. The prior time my wife started it was ten days ago, and that time like this time it behaved the same way: a few minutes of near-silent, slow clicking, then an awakening and normal operation. We suspect it's on the verge of death. Before going to sleep, I dragged the old drive's folders to the 323, and let it run through its 10 hour transfer from sinking skiff to reliable coast guard cutter.

What's the problem with standard USB drives, and why should you not rely on them? Every drive fails, and you don't want to backup to a device with a fuse burning toward self destruction. Before moving to the RAID solution and while looking for a replacement for the old USB drive, I realized that every drive Amazon sells, given enough reviews, will have some frightening proportion of customers saying this like "It died after two weeks and I lost all my data! I'm never buying from this company again! Avoid this drive!" Every drive does this. Really?

I did a little research, and I found a great paper put together by some Google engineers. The guys who support the Google infrastructure have to buy a lot of drives and must know something about failure rates, right? Failure Trends in a Large Disk Drive Population may be more scientific than you're interested in, but at least consider this picture:

drive-failure.jpgEnough said.

Anyway, that's my personal tech journal for the week.

Enjoy!

[Added Mar 30, 2011]

How to Configure Email Alerts
(or How to Crash the Administrative Website of the DNS-323)


Beware of email alert testing. I found a strange bug in the DNS-323 Firmware version 1.09 today. If I use correct email settings, I can send test messages, and alerts work. If I use normal incorrect settings, then I get failures. But if I use special incorrect settings, I crash the administrative web server, essentially freezing it, perhaps due to a blocked process.

I tried several configurations. In each case, I included these settings:

username: myaccount@gmail.com
password: magicpassword
sender email: myaccount@gmail.com
receiver email: myaccount@gmail.com
smtp authentication: check


Proper settings finished with this:

smtp server: smtp.gmail.com, port: 587


Settings that resulted in failure finished with this:

smtp server: smtp.googlemail.com, port: 587

Settings that resulted in a crash finished with either of these:

smtp server: smtp.googlemail.com, port: 465
smtp server: smtp.gmail.com, port: 465


I'm surprised to find that these result in a crash, since the settings were taken from Google's documentation of how to set up outgoing email.

The crashed web server won't come up until it is restarted. Most people will need to reboot the DNS 323 for this. Those of us with ffp installed and ssh access can go in and restart it as follows:

# killall webs
# /web/webs &

By the way, I've seen many people complain in online forums that no logging is available to inform them of what causes their test to fail. If you can log in through ssh and watch the console, then you'll see the errors in the output. For example, with smtp.google.com.com on 587, I saw this:

msmtp: the server does not support authentication
msmtp: could not send mail (account default from /etc/.msmtprc)
Error sending message, child exited 69 (Service unavailable.).


And when I tried testing with smtp.gmail.com on 587 but without checking the box for smtp authentication, I saw this:

msmtp: envelope from address myaccount@gmail.com not accepted by the server
msmtp: server message: 530 5.7.0 Must issue a STARTTLS command first. c18sm589580ana.27
msmtp: could not send mail (account default from /etc/.msmtprc)
Error sending message, child exited 65 (Data format error.).
Mail Server test fail


However, the only way I know to get these in my output is to first restart the web server from the console. I suppose this makes the process of my ssh session the parent to the web server process.

On the crash, no message is sent to output. Instead, the system freezes.

Interesting!

amistad.jpg
Here's a post that will be of little interest to my normal readers but that may be helpful to Googlers. If this helps you, please drop a comment letting me know. I need encouragement to go so far off topic from my normal posts.

Several years ago I bought the history book on which the movie Amistad had been based. The Amistad was a ship carrying slaves to the Americans, and its captives revolted. The movie, which I didn't see, was apparently exciting enough, but the book was tedious. I wanted to never revisit it or anything like it again. But alas, I've encountered what could be called a slave revolt.

MySQL has a strange behavior on slaves with the CHANGE MASTER command that cost me a few hours of sleep. Sometimes when values are set with the command, those values merge into the master.info file. However in other cases after using the command, the values in master.info are lost. A sequence of commands that seemed reasonable to me left me without the proper master bin-log and offset log position, and this caused my slave to get errors like 'Duplicate entry for key 1.'

Here's how I discovered this behavior:

First, I created a dump using the syntax that places within the dump an update statement to set the master's position:

mysqldump --all-databases --master-data=1 --add-locks -u myuser -p > full.db.`date +"%F"`.dmp

Afterward, I can check my dump and find that indeed, it provides the master's bin-log and position:

CHANGE MASTER TO MASTER_LOG_FILE='bin-log.000494', MASTER_LOG_POS=169;

I then bring the dump to my slave server. If I first import the dump and rely on its values to set the master's position, I'll get errors when replication begins. The errors are caused because the replication picks up at the oldest bin-log instead of the right one. The errors, found after running "show slave status\G;" are like this:

Last_Error: Error 'Duplicate entry '3363837' for key 1' on query. Default database: 'myapp'. Query: 'INSERT INTO mytable (
                                    blah,
                                    blah2,
                                    blah3
                                ) VALUES(
                                    '1',
                                    '2009-11-01T00:06:16-05:00',
                                    'stuff'
                                )'

                                
What I really should have done to avoid the errors would have been to run a CHANGE MASTER command that stated everything rather than skipping the details that the dump included.

After looking into this further, I find that as expected, the dump creates a master.info file with the master's proper bin-log and offset, and that master.info doesn't yet have the server connection details. Then after providing just the connection details through CHANGE MASTER, contrary to my expectation it then wipes out the bin-log and offset values rather than properly merging. I can fix this by then providing just the bin-log and offset values, which are properly merged into master.info.

Commands illustrating this are below:

[root@myhost ~]# # import the master's data
[root@myhost ~]# mysql -u root -p{secret} < /tmp/full.db.2009-11-14.dmp
[root@myhost ~]# # see what the dump put into master.info
[root@myhost ~]# cat /var/lib/mysql/master.info # notice this first iteration of the file has no connection info
14
bin-log.000494
169

test

3306
60
0





[root@myhost ~]# # set the partial details as documented
[root@myhost ~]# mysql -u root -p{secret} --execute="CHANGE MASTER TO MASTER_HOST='10.1.1.14', MASTER_PORT=3306, MASTER_USER='repl', MASTER_PASSWORD='supersecret';"
cat /var/lib/mysql/master.info
[root@myhost ~]# # check if that put anything in master.info
[root@myhost ~]# cat /var/lib/mysql/master.info # notice this second iteration dropped the bin-log and log position
14

4
10.1.1.14
repl
supersecret
3306
60
0





[root@myhost ~]# # set the remaining details as though nothing had been in dump
[root@myhost ~]# mysql -u root -p{secret} --execute="CHANGE MASTER TO MASTER_LOG_FILE='bin-log.000494', MASTER_LOG_POS=169;"
[root@myhost ~]# # check if that put anything in master.info
[root@myhost ~]# cat /var/lib/mysql/master.info # notice this third iteration merged in the bin-log and log position
14
bin-log.000494
169
10.1.1.14
repl
supersecret
3306
60
0





[root@myhost ~]# # set everything and see the results:
[root@myhost ~]# mysql -u root -p{secret} --execute="CHANGE MASTER TO MASTER_HOST='10.1.1.14', MASTER_PORT=3306, MASTER_USER='repl', MASTER_PASSWORD='supersecret', MASTER_LOG_FILE='bin-log.000494', MASTER_LOG_POS=169;"
cat /var/lib/mysql/master.info
[root@myhost ~]# cat /var/lib/mysql/master.info # notice this fourth iteration that sets everything looks like the third iteration
14
bin-log.000494
169
10.1.1.14
repl
supersecret
3306
60
0



So in short, don't rely on the dump to set master.info values for you. Just put them all into your mysql prompt similar to this:

mysql> CHANGE MASTER TO MASTER_HOST='10.1.1.14',
    -> MASTER_PORT=3306,
    -> MASTER_USER='repl',
    -> MASTER_PASSWORD='supersecret',
    -> MASTER_LOG_FILE='bin-log.000494',
    -> MASTER_LOG_POS=169;


Enjoy!

A Secret to Install Pandora on Blackberry

| | Comments (5)
pandora_blackberry.pngTo my normal readers, don't worry! This blog will soon return to its regular programming!

While installing Pandora on my Blackberry Pearl, I hit an obscure error message. Google only knew of a single page in the whole wide world wide web that discussed this error, but that page had no solution. So now for the perfect stranger who came here from Google, who doesn't care about Oracle software, who doesn't know me, and who just wants to listen to his or her reggae/polka/rap/country/classical music through the Blackberry, I bring you this special announcement:

Pandora provides different application downloads based on which device it thinks you're using, and it determines your device by your browser's emulation mode. So if you're not using the Blackberry emulation mode, then your device may try to download the wrong one. In my case, I saw the error "Unsupported media type: application/x-cabinet," and the device wanted to download pandora.cab. I was using MS Pocket IE mode when that happened. You may be using a different but still wrong mode, so try setting it properly. To do so:

  1. Open your browser (and it must be the default browser--don't try Opera Mini or some such thing)
  2. Click the Blackberry menu button
  3. Select "Options"
  4. Select "Browser Configuration"
  5. Set your emulation mode to Blackberry
  6. Navigate to http://www.pandora.com (or refresh if you had already been there)
  7. Download, and you'll be in business
  8. For good sound, use a headset of some sort
  9. Give me a comment if this really did help you
Enjoy!