Results tagged “DNS 323” from Bill Benac

How To: Bulk download from Sharepoint

|
This post goes in the "Why wasn't I able to Google that?" category. Remember this old comic?

100322_cartoon_6_a14837_p4651[1].gif
It turns out things that seem like they should be easily Googleable aren't. Maybe this post will be helpful to someone else out there.

I've been helping a small business migrate off Sharepoint and onto a local NAS device (Dlink DNS-323). They have about 4000 documents in 300 folders on Sharepoint hosted by Microsoft Online. How to do a bulk download? The Sharepoint UI (that they hated so much that they asked me if I could migrate them off it) gives no clues. I did searches for these without turning up anything good:

sharepoint "bulk download" "microsoft online"
bulk download sharepoint.microsoftonline.com
sharepoint server 2007 bulk download

Many people offer bulk upload tools, but what about bulk download? Certainly people want to change technologies now and again. I saw one discussion thread that vaguely mentioned WebDAV, but I found nothing about it in the online help, and I found very little for this Google search:

 "microsoft online services" webdav sharepoint

Finally I just gave it my best shot. If it does support WebDAV, which the web hasn't confirmed for me, then how would I go about it?

network-places_smallco.jpgMy laptop runs XP (still my favorite environment), so I used these steps:

  1. My Network Places
  2. Add Network Place
  3. Next
  4. Choose another network location
  5. Enter URL to Sharepoint site: https://smallcomicrosoftonlinecom-2.sharepoint.microsoftonline.com/Shared%20Documents/
  6. Give same credentials used to log into Sharepoint

The laptop of the person for whom I was doing this migration runs Vista, so the process is a little different there. I connected them using notes from a forum:

  1. Hit start menu and go to "Network"
  2. Hit Alt-button to get the tools-menu.
  3. Go to Tools -> Map Network Drive
  4. Click on the link on the bottom that says "Connect to a website that you can use to store your documents and pictures"
  5. Hit Next
  6. Choose "Choose a custom network location" and hit Next.
  7. Enter your url location...

Well, the instructions from here weren't a perfect fit, but on Vista, basically, enter the URL to Sharepoint, then the credentials, then maybe select the drive this will map to.

After connecting by WebDAV, I was able to access the entire Sharepoint site as a folder in Windows Explorer, and I could then open that folder and copy its entire contents to a local disk drive. I brought it to the laptop first, then I copied it onto the NAS drive.

At this point, the folks I'm helping out were able to disable Sharepoint logins for everyone but the administrator. They'll let their Sharepoint subscription lapse at the end of the month, which they're very happy about.

By the way, the DNS-323 that I wrote about and to which I gave a glowing review? It has extraordinarily frustrating user and group management through the web interface. Users can only belong to a single group. So if you want to have a group for managers who would access the /managers share, then a group for accounting who would access the /accounting share? You can't do it with the standard UI. As soon as you add the manager Adam to the accounting group, he is taken out of the manager group. I wound up having to telnet into the server to edit the undocumented, non-standard config file (/mnt/HD_a4/.systemfile/.smb.ses). I don't recommend this NAS for use outside the home.

Love at First Boot: The D-Link DNS 323 NAS

|
D-Link_DNS-323[1].jpg[Added Dec 29 2011] The best love endures overtime. My infatuation has passed. I cannot recommend this device for use outside the home. The default software only lets a user belong to a single group (e.g. finance) but not two (e.g. finance and managers). You can hack around this by using ffp then editing smb.conf, but you don't want to do that routinely. Also, ffp isn't compatible with the most recent firmware, and only the most recent firmware lets you do offsite backup with the CTERA plugin. Also, it requires SMB access requires old and insecure lanman auth, which presents problems when a hardened system like recent Ubuntu distributions want to connect (though you can rollback security on Ubuntu to connect). If I had it to do over again, I would buy the QNAP TS-239 Pro Turbo to get to know a more serious device with disk encryption, better backup options outside the device, better user/group management, etc. And now, the original post...




Remember that giddy feeling when in high school you first ate lunch on the grass with that special someone, the object of your springtime infatuation? Ahhh. So sweet. I'm reliving that feeling with my newly installed NAS. I tenderly call her "323" for short, but her parents call her "D-Link DNS-323 2-Bay Network Storage Enclosure." I can see beyond her toaster looks...

I don't blog about every early Christmas present, but this NAS is so geek-winningly hackable, and I wound up doing such a number on my home network for it that I can't help but share the story. This may be helpful to other web wanderers, just as I relied on many blog posts, discussion forums, and so forth to get set up.

Benefits

First the benefits of this relationship:

1. Network Attached Storage -- You know at the office how nice it is to always have access to those never-ending shared drives that corporate IT provides. I now have it at home. Instead of keeping only select music stored on my computer and the rest locked away on that external USB drive in my wife's office, it's all available. The old cables and plugs were a barrier to access.

2. Peace of Mind -- With RAID-1 and two SATA drives, my data won't get lost when a hard drive fails.  And every hard drive fails sooner or later.

3. Openness -- The 323 runs an embedded Linux, and D-Link built a hook to let folks access the core. Extend it with Subversion, SSH, MySQL, or if you're crazy enough you can even install a new Debian.

4. FTP -- The built-in FTP server and granular security model lets me access, share, or backup content from outside the home.

5. iTunes Server -- The device can discover its music then broadcast it to iTunes clients on the network.

6. Scheduled Downloads -- It can schedule downloads of files and folders from an FTP server, web server, or local network share. I don't want to fully rely on my web hosting providers to backup my data, and this lets me keep a copy too.

The feature list is rich, but not all of it applies to me -- yet. We'll see how my thinking shifts as she and I get to know each other better. Other people though are interested in its BitTorrent feature, UPnP AV server, or others

Now for some details.

The Hard Drives

Kermit[1].jpgIn keeping with the do-it-yourself offering, the 323 doesn't come with hard drives. It's just an enclosure. So what did I buy? I admit that I was driven by price rather than features, but I still wound up with a pair great drives. Amazon was selling Western Digital's energy efficient WD10EADS drives cheaper than any of the other 1TB options, at least with 7200 rpm. It's cool to be green (no matter what Kermit says), but I'm more excited about the cool temperature than the green energy savings. As drives heat up, the probability of failure increases dramatically. More on failures later. The 323 has a feature to monitor the temperature and at high levels, send an email alert and then shut down. I want this feature, but I also don't want it to ever be triggered. The drives were $69 each when I bought, but perhaps for the holidays they have since risen to $84.

Installing the drives to the 323 was easy. I just tore open the drive packages from Amazon, slid the front plate off the 323, and pushed in the drives. No tools required.

Improving the Home Network

In order for my wife and me to share the NAS, our laptops need IP addresses from the same network. Previously, we didn't have this. The Internet drop and primary wireless router (an old WRT54g) are in my office. Since her office is on the other side of the house, and since the house has built-in ethernet wiring from the location of the Internet drop, we put a secondary wireless router (an older BEFW11S4) near her office that pulls data from the ethernet port.That router though was configured the easy way, with DHCP enabled and placing her on a different network. I was on 10.1.10.x, and she was on 10.1.11.x. So here's what I did:

1. Made sure the primary router ran normally, with our ISP's Internet provided through the router's WAN port
2. Made sure the secondary router ran normally, with the primary router's Internet provided through the secondary router's WAN port
3. Changed the secondary router to use a static IP, which in retrospect may not have been necessary
4. Configured the secondary router's Setup->Advanced Routing page to both send and receive RIP 1, which may not have been necessary but one blog suggested
5. Moved the secondary router's ethernet cable from the WAN port to port 1 which does uplink.

That was it. Now when my wife connects to the secondary router by her office, it acts as a switch to get to the primary router, and the primary router gives her an IP address in 10.1.10.x so we can both communicate with the NAS.

Improving the 323

Out of the box the 323 is nice, but it really starts to get cool once you start treating it as a customizable Linux box instead of just a hard drive. The device has a thriving community supporting it, and it's a great example of how a company's decision to open their product up can improve its usefulness and cultivate buzz (e.g. this blog post). The best site for the product may be http://wiki.dns323.info/. I proceeded cautiously installing my first "fun_plug" file to execute my commands at startup based on instructions at that site. Once my feet were wet, I installed a package of Unix tools called "ffp" (Fonz fun_plug) following the instructions at nas-tweaks.net.

In no time at all, I had logged in through telnet, disabled that insecure service, set up SSH, and begun looking around. I then followed the instructions at another blog to install the usb-storage.ko module allowing me to mount Fat32 USB devices through the 323's USB port. I got my wife's old iPod Mini loaded up with little effort.

Breaking Up, not Backing Up, with Standard USB Hard Drives

After setting up a few directories on the hard drives with proper security, I powered up the old 250 gb USB drive that started all this. The prior time my wife started it was ten days ago, and that time like this time it behaved the same way: a few minutes of near-silent, slow clicking, then an awakening and normal operation. We suspect it's on the verge of death. Before going to sleep, I dragged the old drive's folders to the 323, and let it run through its 10 hour transfer from sinking skiff to reliable coast guard cutter.

What's the problem with standard USB drives, and why should you not rely on them? Every drive fails, and you don't want to backup to a device with a fuse burning toward self destruction. Before moving to the RAID solution and while looking for a replacement for the old USB drive, I realized that every drive Amazon sells, given enough reviews, will have some frightening proportion of customers saying this like "It died after two weeks and I lost all my data! I'm never buying from this company again! Avoid this drive!" Every drive does this. Really?

I did a little research, and I found a great paper put together by some Google engineers. The guys who support the Google infrastructure have to buy a lot of drives and must know something about failure rates, right? Failure Trends in a Large Disk Drive Population may be more scientific than you're interested in, but at least consider this picture:

drive-failure.jpgEnough said.

Anyway, that's my personal tech journal for the week.

Enjoy!

[Added Mar 30, 2011]

How to Configure Email Alerts
(or How to Crash the Administrative Website of the DNS-323)


Beware of email alert testing. I found a strange bug in the DNS-323 Firmware version 1.09 today. If I use correct email settings, I can send test messages, and alerts work. If I use normal incorrect settings, then I get failures. But if I use special incorrect settings, I crash the administrative web server, essentially freezing it, perhaps due to a blocked process.

I tried several configurations. In each case, I included these settings:

username: myaccount@gmail.com
password: magicpassword
sender email: myaccount@gmail.com
receiver email: myaccount@gmail.com
smtp authentication: check


Proper settings finished with this:

smtp server: smtp.gmail.com, port: 587


Settings that resulted in failure finished with this:

smtp server: smtp.googlemail.com, port: 587

Settings that resulted in a crash finished with either of these:

smtp server: smtp.googlemail.com, port: 465
smtp server: smtp.gmail.com, port: 465


I'm surprised to find that these result in a crash, since the settings were taken from Google's documentation of how to set up outgoing email.

The crashed web server won't come up until it is restarted. Most people will need to reboot the DNS 323 for this. Those of us with ffp installed and ssh access can go in and restart it as follows:

# killall webs
# /web/webs &

By the way, I've seen many people complain in online forums that no logging is available to inform them of what causes their test to fail. If you can log in through ssh and watch the console, then you'll see the errors in the output. For example, with smtp.google.com.com on 587, I saw this:

msmtp: the server does not support authentication
msmtp: could not send mail (account default from /etc/.msmtprc)
Error sending message, child exited 69 (Service unavailable.).


And when I tried testing with smtp.gmail.com on 587 but without checking the box for smtp authentication, I saw this:

msmtp: envelope from address myaccount@gmail.com not accepted by the server
msmtp: server message: 530 5.7.0 Must issue a STARTTLS command first. c18sm589580ana.27
msmtp: could not send mail (account default from /etc/.msmtprc)
Error sending message, child exited 65 (Data format error.).
Mail Server test fail


However, the only way I know to get these in my output is to first restart the web server from the console. I suppose this makes the process of my ssh session the parent to the web server process.

On the crash, no message is sent to output. Instead, the system freezes.

Interesting!

Find recent content on the main index or look in the archives to find all content.