North Star Labs

I originally installed OpenSolaris 2008.11 on my home media server, and 2009.06 has some features I’d like to take advantage of. Running a pkg image-update, I ran my root pool out of space since it’s located on a relatively small compact flash card.

I decided to grow the root pool by using an available external disk I have. The process involves attaching the new, larger disk to the root pool as a mirror, waiting for the resilver process to complete, installing the boot loader onto the new disk, then detaching the old, small device from the root pool. This information is documented at sun.com in the document How to Replace a Disk in the ZFS Root Pool.

Attempting to attach the new device to the pool with zpool attach, I ran in the error message "cannot label 'c3t0d0': EFI labeled devices are not supported on root pools." I tried wiping the EFI label, but kept running into the same error. I noticed other people talking about this issue;
Removing EFI (format -e not working?) and
Please help need to remove EFI label: msg#00173

My problem was that I was not properly creating the root partition on the disk with an SMI label. I was properly using format -e, then executing “fdisk”, creating the VTOC on the entire disk, but I forgot the step where once the VTOC is created, you need to create partition 0, which will be used for the zpool vdev.

If you run into this error, make sure you use the “partition” option in fdisk -e, which will allow you to then define slice 0. Label the slice “root” and give it as much space as you’d like. Make sure it does not overlap with the boot slice which is automatically created when the VTOC is created.

Once slice 0 is present, use c1t0d0s0 rather than c1t0d0 when you attach the new disk to the root pool. For example:

Correct:
zpool attach rpool c4t0d0s0 c3t0d0s0
Incorrect:
zpool attach rpool c4t0d0s0 c3t0d0
If you receive an error about overlapping partitions, just use zpool attach -f to force the attach.

Once the device is in the pool and re-silvering, use installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c3t0d0s0 to install the boot block.

After testing the new boot device, use zpool detatch rpool c4t0d0s0 to remove the old device from the pool and complete the resize process.

Here’s my original partition layout:

Current partition table (original):
Total disk cylinders available: 3820 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders        Size            Blocks
  0       root    wm       1 - 3818        7.46GB    (3818/0/0) 15638528
  1 unassigned    wm       0               0         (0/0/0)           0
  2     backup    wu       0 - 3819        7.46GB    (3820/0/0) 15646720
  3 unassigned    wm       0               0         (0/0/0)           0
  4 unassigned    wm       0               0         (0/0/0)           0
  5 unassigned    wm       0               0         (0/0/0)           0
  6 unassigned    wm       0               0         (0/0/0)           0
  7 unassigned    wm       0               0         (0/0/0)           0
  8       boot    wu       0 -    0        2.00MB    (1/0/0)        4096
  9 unassigned    wm       0               0         (0/0/0)           0

Here’s my new, larger disk layout:

Current partition table (original):
Total disk cylinders available: 60797 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       boot    wm       1 - 60700      929.97GB    (60700/0/0) 1950291000
  1 unassigned    wm       0                0         (0/0/0)              0
  2     backup    wu       0 - 60796      931.46GB    (60797/0/0) 1953407610
  3 unassigned    wm       0                0         (0/0/0)              0
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6 unassigned    wm       0                0         (0/0/0)              0
  7 unassigned    wm       0                0         (0/0/0)              0
  8       boot    wu       0 -     0       15.69MB    (1/0/0)          32130
  9 unassigned    wm       0                0         (0/0/0)              0

After detaching the original, small disk from the mirror, the root pool expands to the size of the remaining vdev:

jmccune@rain:~$ zpool list rpool
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool   930G  5.57G   924G     0%  ONLINE  -

I cooked dinner for myself today and sat down at the table looking forward to streaming The Daily Show or The Colbert Report on Hulu since there’s no way I’m paying $65 a month for cable TV. As it turns out, there haven’t been any new episodes in awhile, and I like my fake news fresh off the wire, so I decided to catch up on my other fake news addiction; aggregated blog rss feeds.

I came across the TED talk for today, which is Michell Obama. Great speech by the way, check it out at: http://www.ted.com/talks/michelle_obama.html. The streaming video quality left something to be desired, so I looked around and found the HD podcast URL at podcasters.tv.

This works well with iTunes, and MediaLink is able to copy the movie file from my MacBook pro, but for some reason streaming the video usually quits part way through playback with an obscure error code.

I have my OpenSolaris, Intel Atom based file server running on a gigabit network connected up to the Playstation 3 and HDTV using MediaLink, so I decided to look for some unix tool to download the podcast which could easily be run from cron.

After some searching and research into different options, I downloaded Peapod, a wonderful python command line application, and gave it a whirl.

To my complete satisfaction, peapod runs from my home directory without requiring any piece of itself being installed into the system. The only missing dependency I ran into was urlgrabber for python 2.4. Luckily, I have easy_install installed so it was a simple matter of:
pfexec /usr/bin/easy_install-2.4 urlgrabber
Once urlgrabber was installed, setup of the podcast client was a breeze:

jmccune@rain:~$ cd ~/bin
jmccune@rain:~/bin/$ ln -s ../apps/peapod/peapod.py peapod
jmccune@rain:~/bin/$ cd ~
jmccune@rain:~$ peapod
Creating user directory: /home/jmccune/.peapod
Created a default configuration file in :
/home/jmccune/.peapod/peapod.xml
Please edit this file to contain your feeds and options.

I commented out the sample podcast and added TED in HD.
(Note: I found the feed URL by doing a “Get Info”, or clicking on the little i next to the podcast title in the podcast section of iTunes.)

For the title I made it “TED Talks (HD)” and for the URL, I used http://feeds.feedburner.com/TedtalksHD.

Finally, running peapod simply works.

jmccune@rain:~$ peapod
...Spawning thread 0 for feed url http://feeds.feedburner.com/TedtalksHD
Fetching feed for TED Talks (HD)
Downloading TED Talks (HD) -- http://video.ted.com/talks/podcast/MichelleObama_2009P_480.mp4
Trackname MichelleObama_2009P_480.mp4
Savename /export/dozer/podcasts/jmccune/TED Talks (HD)/MichelleObama_2009P_480.mp4
Mime-type video/mp4


This will be added to cron to run every day a few hours before I get home from work, and MediaTomb should pickup the new content.

And now to figure out how to manually kick off a MediaTomb scan of the folder once downloading is complete.

Here are some decent feeds I’ve found so far:

peapod --addnew=http://www.hbo.com/podcasts/billmaher/podcast.xml --title="Bill Maher"


Please post more video feed URL’s, especially 720p and higher, in the comments if you have some good video podcasts worth watching on my TV.

Using OpenSolaris 2008.11, it appears the most complete way to obtain a full-featured development tool chain is to install the ss-dev and gcc-dev package clusters.

pfexec pkg install gcc-dev ss-dev

References:

• Sun Studio Express 11/08 Packages

I spent the better part of the evening attempting to get a reliable, responsive and otherwise unobtrusive DLNA media server running on my new OpenSolaris home file server. I finally stumbled upon PMS which “just works” after using X11 forwarding over ssh once in order to get at the GUI configuration screen. I went ahead and tried the Linux tarball. None of the included binaries execute on solaris, but the jar file appears to run great.

Once running, my PS3 sees the media player quickly and easily and streams my MP3’s nicely.

I’m planning on cooking up a SMF profile to keep this running as a service and figure out the mplayer calls in order to stream my favorite web streams directly to the PS3.

Other DLNA media servers I tried were Coherence (no documentation, didn’t work out of box), Mediatomb (needed to hack the source to get it to run on solaris. When running, rarely showed up in the XMB), fuppes (compile issues).

Contrary to recommendations from seasoned Unix admins, it’s perfectly acceptable to change the root shell from the bourne shell to something like bash. The most common reason to leave the root shell alone usually goes something like, “you need a valid and statically linked shell defined in /etc/passwd to boot into single user mode if you need to recover your system.”

There’s a really nice list of Solaris root shell misconceptions published at http://www.roble.com/docs/sol_root_shell.html.

Fortunately for me, this isn’t the case in Solaris 10. While setting up a new Solaris 10 system today, I accidentally set root’s shell to /sbin/bash instead of /usr/bin/bash. /sbin/bash doesn’t exist, so I could no longer log into the system.

Luckily, this is a system with a Dell RAC card setup for remote console access. I logged into the RAC and issued a “graceful shutdown” power off command, which Solaris responded to nicely and brought the system entirely down. Once I powered the system back on, it’s simply a matter of booting into single user mode by passing the -s flag to the kernel.

Solaris 10 is smart enough to fall back to /sbin/sh if it can’t invoke the shell defined in /etc/passwd booted into single user mode. So long as you don’t horribly mangle /sbin/sh and the libraries it’s linked to, you’ll be fine changing the root shell to anything you like.

Here’s how it went:

Today, I setup my new Atom 330 based OpenSolaris 2008.11 file server to share files using windows file sharing. Windows file sharing uses the SMB/CIFS protocol and is commonly implemented using Samba on Unix. With OpenSolaris 2008.11 and zfs, however, the SMB/CIFS protocol is implemented in the kernel itself by way of an SMB module.

Unfortunately, it’s not quite as simple as executing zfs set sharesmb=on dpool/export/dozer. The SUNWsmbs and SUNWsmbskr packages need to be installed, the system needs to be rebooted, pam configured to create smb password hashes, passwords reset, and finally the smb SMF service needs to be enabled.

You may need to create new filesystems with the casesensitivity and nbmand zfs properties set correct and copy your data over to these filesystems.

Here is the transcript:

pfexec pkg install SUNWsmbs
pfexec pkg install SUNWsmbskr
pfexec reboot
pfexec svcadm enable -r smb/server
pfexec zfs create -o casesensitivity=mixed -o nbmand=on dpool1/export/dozer
pfexec zfs set sharesmb=on dpool1/export/dozer
pfexec bash -c "echo 'other password required pam_smb_passwd.so.1 nowarn' >> /etc/pam.conf"
pfexec passwd jmccune

You must reset your password to generate the new SMB hash value. All users that need SMB access will need to reset their password in this manner.

You may check the sharing status with sharemgr;

jmccune@rain:/export/dozer/isos$ pfexec sharemgr show -vp
default nfs=()
zfs
zfs/dpool1/export/dozer smb=()
dpool1_export_dozer=/export/dozer
dpool1_export_dozer_documents=/export/dozer/documents
dpool1_export_dozer_isos=/export/dozer/isos
dpool1_export_dozer_movies=/export/dozer/movies
dpool1_export_dozer_music=/export/dozer/music
dpool1_export_dozer_pictures=/export/dozer/pictures


Troubleshooting.
svcadm may speak up about a dependency on the physical network. This does not appear to be an error.

jmccune@rain:~$ pfexec svcadm enable -r smb/server
svcadm: svc:/milestone/network depends on svc:/network/physical, which has multiple instances.


You may receive an error that sharing failed. In order to resolve, make sure you’ve done everything I listed above and rebooted the system.

cannot share 'pool/media': smb add share failed


References

• http://www.aspdeveloper.net/tiki-index.php?page=SolarisSMBShareSetup

 

My laptop at work is not automatically backed up, so I maintain a folder named “Warehouse” in My Documents that I used offline files to synchronize to our central file servers, which are backed up.  Since the primary method of accessing offline files is through a mapped drive letter, I had performance issues with shortcuts and applications being accessed through the network.

 

I’ve since disabled offline files, installed the Microsoft SyncToy 2.0 [1] and configured a scheduled task to synchronize the Warehouse folder to the network file server and to a TrueCrypt encrypted disk image on portable, external storage.

This solution works quite well.  If the encrypted disk image is not mounted, SyncToy doesn’t complain, and when I’m off the corporate network there aren’t any issues either.

In order to configure the task to run automatically, make sure to Run “C:Program FilesSyncToy 2.0SyncToyCmd.exe” -R

I have the task run after 10 minutes of idle time, which is perfect for my lunch break or whenever I step away from the computer.  This also prevents lengthy sync processes from kicking off at log-in or log-out which I found annoying with Offline files.

Quick answer:

cfgadm -al

Searching the web for this information took a few more minutes than I expected it to, so I’m posting this article with as may relevant keywords as I can think of. Thanks to Pascal Gienger for the clear answer to this question.

The situation is pretty common for system administrators; you have a production server that’s running out of storage space and you remedy the situation by allocating a new LUN on your back end SAN.

In Linux, I’d typically echo ‘- – -’ > /sys/class/scsi_host/host1/scan in order to issue a rescan, then run multipath -v2, then add the resulting /dev/mpath/foobar device to LVM.

In Solaris 10, this process has been greatly simplified. One command even re-populates your scsi_vhci multipath controller for you.

Consider before scanning:

[jmccune@otto ~]$ sudo format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c4t00D0B2202E001900d0 <DEFAULT cyl 8352 alt 2 hd 255 sec 63>
          /scsi_vhci/disk@g00d0b2202e001900
Specify disk (enter its number): ^D
[jmccune@otto ~]$

And consider the online re-scan of the fiber channel storage system.

[jmccune@otto ~]$ sudo cfgadm -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c0                             fc-fabric    connected    configured   unknown
c0::212000d0b202e201           disk         connected    configured   unknown
c1                             fc-fabric    connected    configured   unknown
c1::212000d0b202e201           disk         connected    configured   unknown
usb0/1                         unknown      empty        unconfigured ok
usb0/2                         unknown      empty        unconfigured ok
usb1/1                         usb-device   connected    configured   ok
usb1/2                         usb-device   connected    configured   ok
usb2/1                         unknown      empty        unconfigured ok
usb2/2                         unknown      empty        unconfigured ok
usb3/1                         unknown      empty        unconfigured ok
usb3/2                         usb-device   connected    configured   ok
usb3/3                         unknown      empty        unconfigured ok
usb3/4                         unknown      empty        unconfigured ok
usb3/5                         unknown      empty        unconfigured ok
usb3/6                         unknown      empty        unconfigured ok
[jmccune@otto ~]$ sudo format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c4t00D0B2202E000900d0 <DEFAULT cyl 50127 alt 2 hd 255 sec 63>
          /scsi_vhci/disk@g00d0b2202e000900
       1. c4t00D0B2202E001900d0 <DEFAULT cyl 8352 alt 2 hd 255 sec 63>
          /scsi_vhci/disk@g00d0b2202e001900
Specify disk (enter its number): ^D

Creating a new ZFS pool based on this new LUN is easy. Note, the new disk is ID zero from format, not id 1.

[jmccune@otto ~]$ sudo zpool create db1 c4t00D0B2202E000900d0
[jmccune@otto ~]$ zpool list
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
db1     382G   112K   382G     0%  ONLINE  -
rpool  63.5G  38.1G  25.4G    60%  ONLINE  -

Keywords: solaris, sun MPxIO, scsi_vhci, san, lun, attach, online, resize, scan, rescan, luxadm.

Slides from Nigel and my Macworld 2009 presentation are now available online.

Here’s my preferred method of drawing attention to screen elements in technical documentation.

Direct Link: Screen Shot Highlights

iPhone / iPod Direct Video Link

Procedure:

1. Copy window to clipboard with ALT+PrintScreen
2. Paste as a new image into the Gimp with CTRL+SHIFT+V
3. Use the rectangular selection tool to select the regions you want to draw attention to.
4. Feather the selection for effect.
5. Create a drop shadow if desired.
6. Insert a new, totally black layer named mask.
7. Keeping the selection in place, select the mask layer and delete the black pixels, creating a “hole” through the layer to the underlying image of the window.
8. Set the mask layer’s transparency appropriately.
9. Save the image, flattening the layers.
10. Insert the image into your word processor of choice.

The embedded screen cast was created with CamStudio, by converting the resulting AVI into an H.264 AVC MP4 file using the SUPER ffmpeg/x264 front end by eRightSoft.  The embedded player is JW FLV Media Player.  All tools are open source software.