Wednesday, December 14, 2005

installing ruby stuff

Having trouble installing ruby pacakges on ubuntu? Follow the simple advice here and install rubyxxdev.

http://www.hoeg.org/blog/2005/10/24/ruby-gems-and-rails-installation/

Tuesday, November 29, 2005

SMP Works!

Following the instructions here works! You can configure SMP support and by golly, my dual 800 is being put to good use. It scares me how happy that makes me. The custom kernel also allowed me to get the built-in sound support happening. Only remaining task is to figure out why the "live" CD was able to go up to much higher screen resolutions. For now I'm stuck with 1280x1024, but the live CD goes up to 1920xsomething or another.

Using a specialized kernel from ppckernel.org

Hmm, another item on my list of things to do. Tonight I built my first debian kernel, though I haven't installed it. I tried hand compiling without using the debian make-kpkg stuff and got a bunch of weird kernel images I didn't know what to do with. So I went back and did make-kpkg. I'll post results later. I was just wondering how these SMP kernels would work out. Yes, my new obsession will be taking advantage of my dual processors!

Monday, November 28, 2005

Linux kernel building guide

"If you have built your main boot drivers as modules (e.g., SCSI host adapter, filesystem, RAID drivers) then you will need to create an initial RAMdisk image. The initrd is a way of sidestepping the chicken and egg problem of booting -- drivers are needed to load the root filesystem but the filesystem cannot be loaded because the drivers are on the filesystem."

from new Kernel HOWTO

So in my HOWTO, one of the things to explore is why it is cool that we can do our thing without an initrd at all. That sentence above gives us more of an idea. The next thing to pursue is how exactly it works without the initrd, and why you would do one over the other in a non-fw world.

Understanding the yaboot bootstrap process

Again, if you're researching and you don't have access to a debian box, here's some juicy bits, this time from the bootstrap man page that comes with yaboot. I'm including this because it gives a good explanation of OpenFirmware paths, boot partitions, and the bootstrap process in general. It assumes that you are working off of an internal IDE drive like a normal person.


NEWWORLD
The process of booting so called NewWorld PowerMacs from
disk starts with OpenFirmware first attempting to execute
the file specified in the boot-device variable. Unlike
older versions of OpenFirmware the NewWorld version will
not attempt to read a boot sector. By default Open­
Firmware attempts to load a file with HFS file type "tbxi"
in the "blessed" directory from each partition of each
disk OpenFirmware is aware of, the first partition/disk
that is found to be bootable is booted immediately.

Ybin(8) configures a bootstrap partition to pass all of
OpenFirmware's tests to determine if the partition is con­
sidered to be bootable or not. The boot script is given
file type "tbxi" and the root directory is marked as
"blessed", the blessing is important because OpenFirmware
will immediately consider a partition unbootable if no
directory is marked as blessed (you can still manually
execute a loader such as yaboot(8) with OpenFirmware even
without a blessed directory but it will not happen auto­
matically).

The MacOS System Folder is always marked as blessed, this
is required for MacOS as well as OpenFirmware. The MacOS
System Folder also contains its own boot loader which has
the tbxi file type, this makes installing yaboot(8) onto a
MacOS partition is difficult. The only way to install
yaboot(8) on a MacOS boot partition is to modify Open­
Firmware to boot the CHRP script directly. Given this it
is highly recommended that you create a dedicated boot­
strap partition for yaboot(8).

Since OpenFirmware boots the first partition it finds to
be bootable it is important that the bootstrap partition
be first on the disk before any MacOS partition, otherwise
MacOS will be booted instead of a dual boot menu used with
yaboot(8).

The bootstrap partition should also NOT be mountable by
MacOS, the reason is MacOS will (almost always) closely
inspect any blessed directories to make sure its real
MacOS, if it is not satisfied that the contents are a real
copy of MacOS it will unbless the directory, resulting in
OpenFirmware no longer considering it bootable. The best
way to protect against this is to create the bootstrap
partition with the partition type "Apple_Bootstrap" which
OpenFirmware accepts as a valid HFS partition, but MacOS
will ignore and refuse to mount. The bootstrap partition
need not be any larger then 800K. 800K is the minimum
size of an HFS filesystem, and is much more then enough
for this purpose. You need not, and should not keep ker­
nels on this partition, yaboot(8) will load them from your
ext2fs root partition just fine, as well as from any HFS
or HFS+ partitions (yaboot(8) uses OpenFirmware's HFS+
filesystem support).

To create the bootstrap partition, use GNU parted(8) or
mac-fdisk(8) to create a partiton of type "Apple_Boot­
strap". This is documented better in mac-fdisks-basics
(http://penguinppc.org/usr/ybin/doc/mac-fdisk-
basics.shtml).

The bootstrap need not and should not be mounted anywhere
on your filesystem, especially not on top of /boot.
Yaboot(8) is able to load the kernels from the ext2fs root
partition so that is where they should be kept.

OpenFirmware maintains a hierarchy of all the hardware it
is aware of. To access or specify a boot device you must
use the OpenFirmware path. For example: the path to a
SCSI hard disk partition might look like this:
/pci@80000000/pci-bridge@d/ADPT,2930CU@2/@2:2 . The first
part, pci@80000000, shows that the target device is
accessed through the PCI bus. The next part is the PCI
bridge, the next is the name of the SCSI host adapter
installed (this name is provided by a BootROM on the card
itself), and after that is the SCSI ID number. The colon
delimits the device from partition specification, so the
last 2 means the second partition of this device. After
the partition number we can specify pathnames to files in
two ways: lazy and absolute. The "," delimits the Open­
Firmware path from the location of the bootfile.
",\\:tbxi" specifies the file that has a HFS file type of
"tbxi" in the blessed directory. If there is not blessed
directory this will fail. The second is to specify a
absolute pathname to an arbitrary file on the disk, exam­
ple: 2:,yaboot would load the file named "yaboot" in the
root directory of the filesystem. It is possible to load
files in subdirectories but OpenFirmware does not always
do this reliably, and any special characters such as an
embedded space must be expressed like %20 (for a space)
the directory separator used by OpenFirmware is the back­
slash \. Example: 2:,\boot\yaboot. Determining the Open­
Firmware path to a given device is unfortunately not a
trivial task. If you are using the built in ATA hard disk
you can use the alias "hd:".

Ybin also includes a utility ofpath(8) which can in most
cases find the OpenFirmware device path from a unix device
node (ie /dev/hda2).

In addition to binary executables OpenFirmware can also
execute a CHRP script. This is somewhat similar to a
shell script. A CHRP script is useful to create simple
boot menus, among other things. CHRP scripts are divided
into sections in a way similar to HTML. Here is a basic
example of a CHRP script used as a wrapper to yaboot(8)
(since OpenFirmware will only load a file with type "tbxi"
if it is a CHRP script).

<CHRP-BOOT>
<COMPATIBLE>
MacRISC
</COMPATIBLE>
<DESCRIPTION>
GNU/Linux PowerPC bootloader
</DESCRIPTION>
<BOOT-SCRIPT>
boot hd:,\\yaboot
</BOOT-SCRIPT>
</CHRP-BOOT>


The COMPATIBLE section defines what machines this script
is compatible with, if the machine name encoded into the
ROM does not match one of these entries OpenFirmware will
print out a lot of incomprehensible junk and fail to load
the script. The DESCRIPTION is ignored by OpenFirmware as
far as I know. The BOOT-SCRIPT section is where arbitrary
OpenFirmware Forth commands may go. They are executed the
same way as you would enter them on the OpenFirmware com­
mand line. The entire script is wrapped with the CHRP-
BOOT tags so that such a script may be attached as a
header to a binary file. Much more complicated and elabo­
rate CHRP scripts are possible but that is beyond the
scope of this document.

Ybin as of version 0.17 includes a more robust script that
is automatically configured with the correct OpenFirmware
paths based on /etc/yaboot.conf. This new script need not
and should not be edited by the user.

If you have G4 hardware then your OpenFirmware may already
have a graphical boot selector built in. This selector can
be accessed by holding down the option key when booting
the machine. You should see a screen with buttons for
each bootable partition. The current version (as of
ybin(8) 0.13) of ofboot includes a badge icon, the button
with a penguin icon is your bootstrap partition. If you
decide to use this built in selector you really do not
need to use a CHRP script that provides a boot menu.
Thanks to Nicholas Humfrey for creating the Badge icon.

Research Notes

A well-commented yaboot.conf file.

What exactly is an initrd file? From the mkinitrd man page:

mkinitrd creates filesystem images which are suitable for use as Linux initial ramdisk (initrd) images. Such images are often used for preloading the block device modules (such as IDE, SCSI or RAID) which are needed to access the root filesystem. mkinitrd automatically loads IDE modules, all scsi_hostadapter entries in /etc/modules.conf, and raid modules if the system's root partition is on raid, which makes it simple to build and use kernels using modular device
drivers.

Any module options specified in /etc/modules.conf are passed to the modules as they are loaded by the initial ramdisk.




Useful notes from the yaboot.conf man page (which you probably won't have on your Mac OSX system!):

boot=boot-device
Specifies the bootstrap(8) partition ybin(8) will
install the boot loader on. This partition needs
to have an HFS or MSDOS filesystem created on it
(except on IBM hardware). ybin(8) will not create
a filesystem. If a filesystem is not present run
mkofboot(8) instead of ybin(8) for the first time.
The bootstrap(8) partition must have a partition
type of Apple_Bootstrap to prevent MacOS[X] from
mounting it. If MacOS is able to mount the boot­
strap(8) partition it will make it unbootable by
removing the attributes ybin(8) set to make the
partition bootable by OpenFirmware. The bootstrap
partition must not be mounted anywhere on your
filesystem, ybin(8) and mkofboot(8) will refuse to
operate on it if it is mounted. On IBM hardware
the bootstrap(8) should be a type 0x41 PReP Boot
partition.

ofboot=of-path
This option defines the OpenFirmware device path to
the bootstrap(8) partition. This is needed so the
first stage ofboot.b loader can be configured prop­
erly. It should include the OpenFirmware path
including the partition number (but not a file­
name). Example: if your bootstrap partition is
/dev/hda2 the OF path will likely be hd:2. As of
ybin(8) 0.18 you no longer are required to (and
should not) specify this option. If left undefined
ybin(8) will attempt to figure out the OpenFirmware
path automatically using the ofpath(8) utility.
You should only need to define this option if
ofpath(8) fails.

magicboot=magicboot-script
Specifies the path to an OpenFirmware CHRP script
that ybin(8) will copy to the bootstrap(8) parti­
tion. Such a script contains Forth commands that
can allow you to do interesting things such as cre­
ating a boot menu to choose between MacOS or
GNU/Linux. Currently you must use a magicboot
script do to a bug in OpenFirmware. Eventually a
CHRP header could be added to the yaboot(8) exe­
cutable so this can be optional. When this option
is defined the magicboot script will be executed by
OF automatically at boot (instead of the install
file.) See man bootstrap(8) for more information
on this. As of ybin 0.22 you should set this to
/usr/local/lib/yaboot/ofboot which is a autoconfig­
uring first stage loader for yaboot. It is capable
of presenting a dual boot menu for GNU/Linux, MacOS
and MacOSX. If dual booting is not required or
configured it will simply load yaboot directly.
You must specify this for most PowerMacs or they
will fail to boot. You cannot use magicboot
scripts with IBM hardware.

Thursday, November 24, 2005

NAILED: HOWTO install bootable mac ubuntu on your external firewire harddrive

This is version 0.1 of this document.

Purpose and Rationale

To explain to you, the eager Mac linux noobie, how to get your Ubuntu system of your dreams up and running. I am performing the surgery on a dual 800 G4 mac; YMMV. The idea is to be able to install Ubuntu onto an external firewire drive so you don't have to trash your existing Mac OS X set-up. I use my Mac to run Logic Pro, so I had no interest in re-partitioning my internal hard drives. I am also a software developer and wanted to be able to use familiar Linux tools and not have to mess with weird Darwin/BSD hack arounds for my projects.

Known Issues

I don't currently have instructions for how to download, build, and install a debian linux kernel with the SCSI stuff compiled in (rather than built as a module). Thanks and praise are due to Brent Stephens who hooked me up with the deb packages I needed. I am a Redhat hacker by trade, so I am new to all this debian stuff.

I plan on creating a condensed HOWTO to explain how to download, build, and install a debian linux kernel at the "yaboot install failure" juncture. For now, you'll have to rely upon the kindliness of strangers to provide you with the necessary

Background

A quick and dirty explanation of why you can't just install Ubuntu to your firewire drive. Nerds, feel free to correct my crude explanations. Be sure to flame me for how stupid I am to further enhance my impression of engineering culture. No cynicism here, no siree bob.

I know this explanation is crap, but it gives a high level explanation of some kind. Ubuntu is designed to work with a kernel image. The kernel image lives on your hard drive. In order to access your hard drive, the bootstrap process needs to be able to access the SCSI drive on which your kernel lives. If the SCSI code for the kernel lives in a module, the kernel image won't be able to find itself, and you're screwed.

The bootstrap thingy is called yaboot. It creates a Mac-readable partition known as a bootstrap partition. It has special magic powers that enable it to find a linux kernel image when you provide it with the path to the image. Apple uses something called Open Firmware, which is a very rough and tumble boot loader among other things. One tricky bit about this whole business is that you have to provide yaboot with Open Firmware paths to your image, which is a relatively obscure activity that the Ubuntu installer doesn't know how to deal with. So it will die when it tries to perform the sacred ritual known as installing yaboot. Ubuntu just uses plain old /dev/sdX, and mkofboot needs Open Firmware (I'll refer to it as OF) paths in order to work.

Another note is that the usual way to boot up linux is to use an initrd, which I think is a bootstrap kernel that gets loaded into memory, which then finds your real kernel (/boot/vmlinux). What we're doing here doesn't require initrd, which maybe one day we'll both understand how cool that is.

Make it work for me dammit! I want my Mommy

You need to roll your own yaboot.conf and install it, but that's easier said then done. Here's some step-by-step with some generous hand holding. Be sure to do some t'ai ch'i breathing and review your typing carefully. Welcome to the real world, boys and girls. Btw, once you get this thing up and running, you'll never have to deal with this crap again. You can happily use all the friendly Ubuntu GUI stuff afterwards.


Credit where it's due: Brent pointed out this article which was really helpful, but it is intended for hackers in the know.


      Follow the usual instructions to download and burn an Ubuntu install CD iso.

      Begin the install process by restarting your mac with your iso in the drive. Hold down 'C' to boot off of your CD-ROM.

      Go about the installation tasks as usual -- follow the wiki above if need be. Be sure to make a note of the BOOTSTRAP partition and the ROOT partition. For me, these were /dev/sda2 and /dev/sda3, respectively.

      Your install will crap out when it tries to install yaboot. Now the fun begins. Drop to a terminal by typing "alt-f2" (aka option-f2).

      Note: this doc will be updated since we can't count on Brent Stephens to leave these deb packages lying around forever. I'm making these notes for posterity.

      Type chroot /target

      You need to figure out the OF path of your FW drive. Type "echo /proc/device-tree/pci*/firewire*/node*/sbp-2*". Write down what it says carefully and legibly.

      Type "apt-get install wget". Make sure your cdrom is still mounted and all that jazz. If it got ejected, apt-get will magically remount it so don't worry about it.

      Type "cd /tmp", then type "wget http://brent.warehouse6.com/misc/kernel.tbz2"

      Type "tar xvjf kernel.tbz2".

      cd into that dir, kernel-2.6.14.2 or something like that, mine is blown away now

      type "dpkg -i *". you can ignore whatever warning messages safely.

      Now you need to make your yaboot.conf. Use your favorite editor, like vi /etc/yaboot.conf. In the file, put the following:

      boot=/dev/sda2

      Unix path to the bootstrap partition (Note: this will be where ever you put it, as noted above)

      ofboot=fw/node@00d04b4a1905397c/sbp-2@c000/disk@0:2


      Open Firmware path to the bootstrap partition. Remember the output from the "echo /proc/device-tree/..." command? Follow my example above for formatting help. The number 2 in my example is the bootstrap partition. Like /dev/sda2 means use the number 2 at the end. You can ignore everything before the node part from the output of the echo command.

      ofdevice=fw/@00d04b4a1905397c/sbp-2@c000/disk@0:


      Open Firmware path to the root partition, more or less. You specify the actual partition next, this is just the path to the device.

      partition=3


      number of the root partition (like mine is /dev/sda3, so use the number 3)

      Now there's some stock stuff that goes in here which I'm not going to explain. just do it.


      timeout=40
      install=/usr/lib/yaboot/yaboot
      magicboot=/usr/lib/yaboot/ofboot
      default=linux
      macosx=/dev/hda5 --> if you edit /etc/yaboot.conf, you'll find ubuntu's attempt at this file. Find the value for macosx in there. don't just copy what I put.

      image=/boot/vmlinux
      label=Linux
      root=8:3
      append="rootdelay=10"


      Ok, that "root=8:3" part is tricky. Go here to figure out the correct values for your installation.

      My yaboot.conf:


      boot=/dev/sda2
      ofboot=fw/node@00d04b4a1905397c/sbp-2@c000/disk@0:2
      ofdevice=fw/node@00d04b4a1905397c/sbp-2@c000/disk@0:
      partition=3
      timeout=40
      install=/usr/lib/yaboot/yaboot
      magicboot=/usr/lib/yaboot/ofboot
      default=linux
      macosx=/dev/hda5

      image=/boot/vmlinux
      label=Linux
      root=8:3
      append="rootdelay=10"


      Ok, now you must install this gizmo onto your bootstrap partition. Type "mkofboot -v" and hope it works! You should see some promising output.

      Switch back to the install by pressing alt-f1. Bail out of there and restart. Hold down ALT + APPLE KEY + O + F to get to the OF bootloader thingy.

      At the bootloader, type "boot fw/node/sbp-2/disk@0:2, yaboot" substituting YOUR bootstrap partition for the number 2 in the disk@0:2 part.

      Type "mac-boot" or "mac-start" or whatever it says, it tells you in the OF boot thing, I'll look it up later.

      Congrats, you're at the yaboot prompt. The screen will flash and your computer will convulse for a minute. when you get the "boot:" prompt, type Linux and go! The Ubunutu install process will magically continue.