Tony's ramblings on Open Source Software, Life and Photography

Moving ZPOOL to New Controllers Like a Ninja

In an effort to squeeze a bit more performance out of my Gluster / ZFS setup I purchased four PCI-Express low-profile SATA 6G cards for my two servers. Each card will control two of the four drives in the zpool, leaving the SSD boot and SSD cache drives on the motherboard ports. The server I use has all the drives connected to the motherboard.

Migration
Migrating was a fairly simple matter. First, I stopped Gluster on the server, and then I did the critical first step and exported the zpool with:

zpool export data

Next, shut down the server and with a pair of tweezers connect new SATA cables to the hotswap backplane ports and install the new controller cards. Okay, not really with tweezers but dang it was tight in there. I could barely get my fingers behind the backplane. Reboot.

Now, check the status of the zpool to see if we can import it:

zpool import

Freak out because two of the four drives were missing. It turns out when you don't plug a hotswap hard drive entirely into it's slot, it doesn't like to work. Reseat the two drives I'd left half-out of their sockets and try again:

zpool import
zpool import data

Tada, all is well in the world and I've migrated all four drives to new controllers.


ZFS Is 42.

For those who don't get the reference (and I bet of my readers that's very few) 42 is the answer to the secret of life, the universe and everything.

For the past few weeks I've been playing with the ZFS file system on Linux. And I keep upping the ante, so to speak. I started by simply creating a home file server running three 2TB hard drives in a RAIDZ configuration with a 128GB SSD boot drive. RAIDZ is ZFS specific and roughly equivalent to RAID-5.

ZFS is unique in that you don't format your drives like you would with a typical filesystem such as FAT, NTFS or EXT4. You simply give it hard drives and say "here." You can then create logical groups of datasets (directories) within the root of that pool of drives, which comes in handy if you want to use snapshots.

Save that Data!

ZFS was developed from the ground up for data stability. It's disk check utility "scrub" runs on your active, mounted filesystem. With RAIDZ, you get all the advantages of RAID-5 without the headache of proprietary controllers or FAKERAID.

Getting Ready

Here's how you go about installing ZFS for Linux on a recent Ubuntu distribution. First install the prerequisites:

apt-get update
  

Categories:

SSH Hostname Autocompletion

Do you use SSH quite a bit, and grow tired of typing in the machine names each time?

There's an easy fix for that.

First, create a text file in your home directory called "hostfile". Populate it with a list of hosts, prepended by 0.0.0.0 (a bogus IP address.) For instance:

0.0.0.0 laptop1.home.com
0.0.0.0 desktop2.home.com
0.0.0.0 www.myserver.com

Next, add the following line to the bottom of your .bashrc file. Now, the .bashrc file is hidden by default, so if you are browsing to find it you'll need to pick "Show Hidden Files" from the View menu of your file browser.

export HOSTFILE="/home/tony/hostfile"

Obviously replace my name with your home directory name.

Now, you can type "ssh l" and it will automatically suggest "laptop1.home.com"! Quick and easy!


Categories:

LDAP Authentication and NSCD

I don't know how I didn't run into this before, but I finally stumbled on a program that just made my life a ton better.

We run OpenLDAP for network authentication, among other things. Periodically, the Name Service Caching Daemon (NSCD) will introduce a bug that causes accounts stored in LDAP to not work properly. In the most recent iteration of Ubuntu Precise 12.04.2, the bug is that "getent passwd" will list all your users, but trying to su to them will tell you they don't exist.

All your problems can be fixed by simply installing "unscd" instead of "nscd". "unscd" or "Micro Name Service Caching Daemon" (the U stands for the micro symbol) is a direct replacement for nscd that doesn't appear to have the problems of nscd.

So if you're running LDAP on your network, or heck even if you aren't, simply install unscd. When you do, it should automatically remove nscd as you can't run them both at the same time.


Categories:

Formatting Large USB Sticks for MP3 Use

A lot of newer vehicles have USB sockets for plugging in your music collection. The default FAT32 format that Ubuntu does in the disk manager just doesn't cut it. In many cases, the vehicle may not even be able to read a stick formatted with the default Ubuntu method.

So, here's how you do it. Let's say your USB stick is located at /dev/sdc and that it currently is NOT mounted. If it is mounted, be sure to unmount it first.

Warning* This will delete everything on the disk. Make sure you don't need it, and make absolutely certain you enter the correct path to the device (i.e. /dev/sdc or wherever your device is located. Your path will likely be different than mine!)

Try the following:

fdisk /dev/sd[c]

Eliminate the braces above and enter the proper path to your device.

Inside fdisk, do:

  • o - Create a new DOS partition table
  • n - Create a new partition - choose the proper size to fill your device.
  • t - Change the partition type - select "c", which is W95 FAT32 (LBA)
  • w - Write the changes and exit.

It probably wouldn't hurt to remove and reconnect the USB device at this point.

Next, format the partition:

mkfs.vfat /dev/sd[c]1 -s 64 -F 32

Again, replace the braces section above with the proper path to your device.


Ubuntu Quantal policykit failure

So since installing Ubuntu 12.10 Quantal, I've been plagued by strange authentication errors. I thought for some time that it was related to my use of LDAP for domain admin authentication, but it turns out it's more devious.

I run Gnome3. That means when I launch something like Synaptic I typically pop the menu up, type in "syna" and hit enter to launch the app.

The first thing Policykit does is open the password dialog.

I would then type in my password (I'm a fast typist) and hit enter. At this point it rejects my password even when correct.

Turns out, the enter key to launch synaptic is ALSO submitting the empty password dialog most of the time. It's not waiting for a keyup event before interpreting the keydown event on the password dialog, so simply holding the enter key at launch of whatever triggers the password dialog to submit a blank password.

This also occurs when using pkexec from the command line. I've filed a bug report.


Categories:

My new app

After nearly two years of off-and-on development I finally finished it!

MilesTrac Online integrates everything suggested by people who used my old MilesTrac Free app but were beyond the scope of the original design.

MilesTrac Online is a quick and easy mileage log tracking tool for taxes or business reimbursement.
MilesTrac makes it easy to enter odometer readings and destination notes for a Mileage Log. Use our website at https://www.milestraconline.com/ to manage your data and export spreadsheets.

  • Never lose your data - sync's to our cloud servers
  • Sync your trips across multiple devices with one MilesTrac account
  • Retains the last 90 days of trips on your device, others available on the website
  • Handles multiple vehicles
  • Detects which vehicle to start a trip for by using your bluetooth pairing
  • Category and minor expense tracking

Replacing your phone? Don't even worry about backing up your old vehicles, categories or trips. Just log in on the new device and everything gets synchronized automatically.
New accounts get a 30 day trial, after which a USD $12 annual subscription applies.


Finding Packet Loss

Finding Packet loss on a large wide network or within your ISP's network can be difficult. Generally the cause of packet loss is bad hardware or lines at a specific location, or an oversaturated network. Since most pathways on the internet will require 10 hops or more, figuring out which hop is troublesome can be difficult.

Traceroute may help, but if the packet loss is intermittent then you may think the loss is happening further down the line than it is.

Ping is nice, but you'd have to ping each hop repeatedly, starting at the closest and working your way out.

The tool for the job is My traceroute (mtr). It will ping a specified number of times to every hop on the path, and if the path changes - as sometimes they will - it will ping all of the other routers and list them out as alternates.

My default mtr will give you an interactive constantly updating display of the results. You may want to generate a report you can email to your ISP when you complain about the problem however, and there's a few switches that allow you to do that.

  -c X   Sends X pings and then stops
  -r     Generate a report and quit

So the command you would use might look like:

mtr -c 100 -r 74.125.228.17

The Art of Business.

A discussion with my G+ friend Michael Adkins got me thinking about the art of photography as a business. It left me mulling over several thoughts that I wanted to share with anyone who will listen. I've been self-employed (not as a photographer) for nearly 10 years now so I have a bit of experience to speak from. And please, take it all with a grain of salt and realize I am smirking as I write it. :-)

I think much of what I'm going to say here has applications in any line of business, but it came up as a discussion regarding photography specifically.

Good Photography is an Art

I need to make a few statements to ensure we're all on the same page. First, good photography is an art. There are those who will tell you that photography is not an art, and I agree that most photography is not art. However, good photography is an art. It requires a keen perception, a bit of imagination, and a great deal of skill. That said, the current tools have dumbed it down to the point that anyone can take an OK photo.


Categories:

LDAP Authentication Breaks on Ubuntu Distribution Upgrade

I've been fighting with bug #1000205 in the latest LTS Ubuntu 12.04 "Precise". Every time I've done a do-release-upgrade on a server, when it reboots it breaks logins. All my LDAP users are no longer available.

My Puppet configurations deploy the proper config file for nslcd, so when this happens I just have to wait for the server to get the correct config file from Puppet, then manually reboot the box. Unfortunately this requires physical access to the server, and I have servers in offsite locations as well.

I finally figured out an easy fix that I can pre-deploy with Puppet. I just wrote a simple script to replace the nslcd.conf file with a known good copy prior to starting the nslcd daemon.

So, here's the basics. I already had the following in my Puppet deployment:

	file {"/etc/nslcd.conf":
		source => "puppet:///modules/ldapclient/nslcd.conf",
		owner => root,
		group => root,
		mode => 644,
		require => Package["ldap-auth-client"],
	}

So I added a second copy of that file on the systems with:

	file {"/etc/nslcd.keep":
		source => "puppet:///modules/ldapclient/nslcd.conf",
		owner => root,
		group => root,
		mode => 644,
	}

Categories: