October 05, 2006
Note to self: investigate GFS (global file system)
At work, we don't have a very good solution for a big, central, reliable file system. NFS on Linux just plain doesn't work: it craps out randomly. Right now we are using samba, which craps out less randomly, but is still not as reliable as NFS was on Solaris 10 years ago.
Self, you need to look into GFS in more detail, and Linux clustering in general to see if it will help.
Here are some links to get started:
http://www.redhat.com/software/rha/gfs/
http://en.wikipedia.org/wiki/Global_File_System
http://sources.redhat.com/cluster/gfs/
Posted by billo at 10:49 AM | Comments (0)
How to find out what version your red hat-flavored distro is
"uname -a" tells you what kernel you have. But sometimes one needs to know what distribution you are running: Fedora Core X, CentOS X.X, etc.
On Red Hat systems, simply:
% cat /etc/redhat-release CentOS release 4.4 (Final) %
Posted by billo at 08:48 AM | Comments (0)
September 27, 2006
How to setup a local DNS server (idiot's version).
I like to maintain my own DNS on my home network. So I can give all the machines fixed IP addresses and nice names. It makes file sharing between machine easier, and, of course, lets me ssh around more easily.
The problem is that I'm too much of an idiot to figure out how to set up a bind/named config from scratch. I don't do it very often, and each time it seems like bind has radically changed the way it works. So I spend hours cobbling together a named.conf and zone files, and finally it works.
This time, I went looking for a GUI DNS config tool, just to get myself going. I want to get the thing started, then I could edit the config files myself to change and add A records and CNAMES. I ended up using gbindadmin. As a productivity/maintenance tool, it's terrible: you can't change A records or CNAMES once you put them in, for example. But it got my config going in just a few minutes, and that's what I was looking for.
Now that I have a new bind running on my ubuntu machine, I can decommission my ancient slackware server and use it for something else. Or at least wipe it out and bring it up with ubuntu or centos.
Posted by billo at 04:02 PM | Comments (0)
September 13, 2006
Powerbook back to Tiger
Remember how I switched my old powerbook to Ubuntu?
Well, I switched back. The finer points (WPA support, good power management)
just made it not worth it.
I did a clean install of tiger, and turned off spotlight and dashboard. It's
reasonably usable now; I'm going to try hard to avoid installing
anything but the bare minimum on the machine.
So far: only Firefox, Developer Tools and Darwin Ports.
Posted by billo at 02:31 PM | Comments (0)
September 08, 2006
How to use VNC server on CentOS
Assume you have CentOS installed with Gnome. If you want to remotely access the GUI to that machine, you can use vnc-server and a vnc client (like Chicken of the VNC). Here's how:
ssh to the machine. If it is behind a firewall, you might have to set up a tunnel.
Make sure vnc-server is installed:
% sudo yum install vnc-server
Run vnc-server:
% vnc-server
This is just to initially create configs and choose a password. Now Edit the file ~/.vnc/xstartup so it runs a gnome session instead of crappy twm:
#!/bin/sh # Uncomment the following two lines for normal desktop: # unset SESSION_MANAGER # exec /etc/X11/xinit/xinitrc [ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup [ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources xsetroot -solid grey vncconfig -iconic & xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" & exec gnome-session &
Kill off your vnc server and restart it:
% killall Xvnc % vnc-server
Connect to your vnc server with host and display number reported with your favorite client. Yay.
Posted by billo at 11:51 AM | Comments (0)
September 06, 2006
Asterisk Open Source PBX
We bough an inexpensive cheapo PBX for work about two years ago. At the time, it seemed like a great idea. It was way less money than the equivalent Nortel or Lucent office phone system. The office extensions were IP phones, but they were H323 phones, and not SIP phones. It's been really terrible. The phones themselves have terrible build quality, bad buttons and usability, and very, very, very bad sound quality. It was about $12,000 for everything: server, phones and software.
Recently we spent a little money to try out Asterisk, the free, open-source PBX. It's software developed by a company that sells hardware for connecting to phones and phone lines, Digium.
It is really cool, and it works. I set up a CentOS Linux with one of the 24-port Digium boards. All extensions and dialing rules, and IVR trees are managed with .conf files. I used the O'Reilly Asterisk book as documentation, since the documentation that is available on the web is pretty bad and/or hard to find. It took about 6 hours to get everything set up the way I wanted it, including voice mail boxes for everybody, a dial by name directory and a connection to outbound VOIP (so we can use fewer analog lines from Verizon.)
We're at the point now where we are going to buy new SIP phones for everyone (we need about 24 of them in total). The total cash outlay on the product will then be about $5000, including the phones and the server and the digium hardware. We should be able sell our crummy old system for about $3000, if eBay prices for it hold up.
There are some commercial vendors that sell complete, turnkey Asterisk systems. Fonality seems to be the most popular. While I'm sure they are great quality, they aren't really as hackable, since the turnkey vendors can't support people mucking about in the configurations.
It's even tempting to replace my home phone system with an Asterisk one: about $250 for a 4-line board (2 incoming and 2 outgoing), and a couple of SIP phones I could really stop those telemarketers!
Posted by billo at 03:50 PM | Comments (0)
August 28, 2006
How to use an SSH tunnel chain to access a splunk server behind a double firewall.
If you have a server environment that is behind a "jump box," and you have some network services you want to access, you can use SSH tunneling to get to it. In this example, I have a splunk server inside the inner firewall, a jump server in the DMZ (but behind the outer firewall). To access my splunk web UI, I do this:
ssh -t -L 8000:localhost:8000 jumpbox.example.com 'ssh -L 8000:splunk.inside.example.com:8000 splunk01'
This forwards access to http://localhost:8000/ through the chain to my splunk server.
Posted by billo at 02:18 PM | Comments (0)
August 11, 2006
Ubuntu Linux on Powerbook G4
I got sick of how slow my old powerbook (800MHz) G4 was running Mac OS X (Tiger). I wiped out the disk and installed Ubuntu 6.06. It seems to be just working. Sound, graphics, battery management and wireless network all seem to work out of the box. Unfortunately, it doesn't come pre configured with essentials like emacs or tcsh, so I have to add those.
Seems to boot a lot faster, to start!
Update:
Some bad things: sleep/suspend don't seem to actually work; the fan is still going while it's "asleep."
WPA2 encryption is known not to work, apparently, a big/little endian thing in the base driver.
There is no nvidia driver for PPC, only i386.
I'll stick with it for a few weeks to see if the no sleep/no WPA are showstoppers.
Posted by billo at 04:18 PM | Comments (2)
August 10, 2006
note to self: how to become a CentOS mirror
http://www.centos.org/modules/tinycontent/index.php?id=22
Posted by billo at 02:17 PM | Comments (0)
July 23, 2006
Pre-installed linux hardware.
Some companies are trying to sell PCs with pre-installed Linux. Hopefully they fare better than VA Linux did. One that looks interesting is system76.com. They ship with Ubuntu, and have some interesting form factors, including one that is a dead rip-off of the Mac Mini.
My notebook PC (an Apple Powerbook) is getting a bit long in the tooth. It's soooo dog slow, that I would like something snappier. The new Mac Pros are very nice, but so expensive. A nice, light notebook without a Windows tax might be a good alternative.
Posted by billo at 09:52 PM | Comments (1)
July 14, 2006
How to upgrade to a recent FireFox on CentOS 4.3
I like CentOS more and more. Yesterday, I actually set it up as a workstation on a PC instead of a server. In doing so, I discovered that the FireFox that ships with CentOS is really old, like 1.0.7.
To upgrade to the latest thing, you need to enable the "centosplus" repository. Like this:
yum --enablerepo=centosplus upgrade firefox
Posted by billo at 09:33 AM | Comments (1)
July 07, 2006
How to unmount a dead smbfs file system.
Little known fact (to me, at least): if you have a mounted smbfs file system, and the server reboots, then the smbfs will not re-connect with the server. It will be essential a wedged file system, causing all to access it to hang.
"umount -f" does not fix this. Neither does "kill -9" of the mountd process responsible for the FS.
The magic command is:
umount -l /path/to/mount
Posted by billo at 01:45 PM | Comments (0)
samba (smbfs) problems: bad superblock error on mount
At work today, we had this problem, and I wasted a long time trying to figure it out. Basically, we have a machine that serves up a big disk over smb/cifs/netbios/whatever you want to call it. There are something like 120 machines that are supposed to mount this as a file system using the smbfs that is now built in to the Linux kernel. We are using mostly FC3, but are switching over to CentOS, to get a little more stability. Pretty soon we'll have over 200 machines, and it's a pain to upgrade them all.
But the problem was that batches of them would get "bad superblock error or too many mounts" when they tried to mount the file system. At first I thought that there was some quota or limit on the samba server side, and we just had too many clients mounting the file system. That wasn't the problem.
Then I noticed that the failure seemed to correlate with slight variations in kernel on the client side. It wasn't perfect, and it turned out to not be the problem either.
The problem was, in fact, that the smbfs kernel module just does not work if the "samba-client" RPM is not installed on the machine. The kernel module is there. It just needs the client libraries to work. It would have been real nice if the error was "oh gosh, samba library not found" instead of "bad superblock." Oh well. The fix was simple:
yum -y install samba-client
on all the affected machines.
Posted by billo at 01:41 PM | Comments (0)
June 28, 2006
arping
I learned something new today about ARP caches.
I was working on this script for work to move an IP address (alias) from
one physical machine to another. (The way we have things set up is that
each of our physical servers has a primary machine-specific IP address. Each
application instance we run on that machine has its own IP address; so the
way we move an application/service is by simply assigning the IP to
another machine, and installing the files on that other machine.
We think it is a clever way of managing application instances.)
I ran into a problem where the IP would be taken down on machine A,
and successfully added to machine B. But only machines on the same subnet
as machine B would be able to ping the new address. Anything outside
would time out. Like our web servers, which kind of defeats the whole
purpose.
The problem was the ARP cache on the fancy Cisco switches we use.
They would refuse to re-arp when the ping failed. That seems silly
to me, but Cisco knows a lot more about networking than I do, so
I suppose there is a good reason for it.
In any case, our network dude (thanks Chris!) figured out that you can run
a utility on Linux called "arping" to cause the Cisco switch (or
anybody else nearby who has cached the old MAC address) to
invalidate the ARP cache and fix everything. Like this:
/sbin/arping -c 4 -U NEWIPADDR
I run that on machine B, and it clears up the problem. Yay.
Posted by billo at 11:00 AM | Comments (0)
June 15, 2006
Other random checklist items for creating CentOS kickstart server
In an earlier note, I documented how to create a kickstart server to do automated network installs for FC4.
I recently repeated the process for CentOS, and found a few things that I had forgotten about. This is a brief recap.
- You need to make sure you install a DHCP server, a TFTP server and Apache:
yum install tftp-server
yum install dhcp
2. Edit the dhcpd.conf as in the other article.
3. Edit /etc/xinetd.d/tftp and set "disable" to "no"
4. Use this rsync script:
#! /bin/sh mirror=rsync://mirrors.kernel.org/centos localdir=/var/www/html/centos-mirror rsync -azvH --exclude-from=/l/etc/centos-rsync-exclude.txt $mirror $localdir createrepo -q $localdir/4/updates repoview -f -q $localdir/4/updates createrepo -q $localdir/4/os repoview -f -q $localdir/4/os createrepo -q $localdir/4/extras repoview -f -q $localdir/4/extras
This assumes you have repoview utility to make yum repositories web browse-able.
Posted by billo at 05:47 PM | Comments (0)
May 22, 2006
OpenVPN setup; many clients to one server and LAN
I used OpenVPN about 2 years ago to connect our US development office with our engineering lab in Bangalore, India. Unfortunately, the dual-site development thing didn't work out, and I haven't used OpenVPN since. But recently I was faced with creating a secure yet inexpensive (read: free) remote access solution for my wife's medical practice. The doctors are, shall we say, not major computer geeks. So an ssh tunneling system, while it would work, would be too difficult to explain, and be too prone to human error.
I remembered OpenVPN, and thought it might work well. I was heartened to see that OpenVPN now has a bunch of GUI clients, which will make setup a lot easier.
First, I had to set up port forwarding from the Linksys router to the Linux server where I would run the OpenVPN server. This was a simple matter of setting port 1194 to forward from the internet to the internal IP address of the Linux server. Ideally, I would have made the internal network of the office something that would not collide with common home routers (usually 192.168.x.x). But the office net was already set up as 192.168.1.x, so I'm going to have to work around that by changing people's home routers to be something else.
Once that was done, I built openvpn from source, installed on the linux machine and created a local certificate authority. The OpenVPN documentation and scripting in this area has greatly improved in the last two years.
In the server.conf, I enabled the TUN device only, since I'm only dealing with IP protocols. Aside from that, the interesting parts of the server.conf were:
# I picked 10.68.0.0/24 as my VPN subnet, since it is unlikely to collide
# with anything. server 10.68.0.0 255.255.255.0
# allow clients to see the rest of the office subnet
push "route 192.168.1.0 255.255.255.0"
Then I created client config files that match. Interesting parts of the the client.conf:
# this is the outside address of the office
remote office.example.com 1194
# these are the key files. I call them the same thing on all
# client machines; I then give each doc a different client key and cert
ca ca.crt
cert client.crt
key client.key
I used both the OpenVPN GUIs for Macintosh and Windows. They worked great except for one problem: the clients could only see the openvpn server machine and nothing else on the office subnet. I knew it was a routing problem, but I couldn't figure out what.
First I had to put a route on the office Linksys router, so that machines on the subnet would route packets for 10.68/24 to the openvpn server, instead of out the internet. I figured I was home free once I saw that you could actually do this on the linksys.
But no! Packets dropped into the void somewhere between the clients and the LAN machines. I did some more reading of the doc, and found the missing link. And a little light flicked on in my dim noggin. This was the same (and final) problem I had struggled with two years ago. To wit, on the server I needed:
echo 1 > /proc/sys/net/ipv4/ip_forward
Once IP forwarding was enabled, everything worked like a charm.
Posted by billo at 10:30 PM | Comments (0)
May 19, 2006
Enabling Remote X Server access on Ubuntu
I just upgraded my Ubuntu box to the latest beta. I ran into the same problem I ran into when I first installed the last release, and I could not remember the fix. So I'm writing it down so I won't forget. Ubuntu disables network socket connections to the X11 server by default. For "security" reasons. (Um, I thought that's what xhost access control was for...)
Anyway, you can fix the problem by editing /etc/X11/gdm/gdm.conf and changing
DisallowTCP=true
to
DisallowTCP=false
Posted by billo at 09:29 AM | Comments (1)
April 07, 2006
notes on building PHP 5 on a Fedora Core 3 system
I have a generally vanilla FC3 system. I wanted to update to PHP 5 because of security issues in PHP. These are the extra things I had to do to get it to work.
First, my configure line is this:
./configure --with-apxs2=/usr/sbin/apxs --with-mysql
Problems:
I had to make links for libjpeg and libpng:
cd /usr/lib
ln -s libjpeg.so.6 libjpeg.so
ln -s libpng.so.3 libpng.so
These are packages I needed to add:
yum -y install libpng-devel
yum -y install freetype-devel
yum -y install libc-client-devel
yum -y install unixODBC-devel
yum -y install aspell-devel
I actually had some trouble with postgresSQL connector, so I turned it off. I don't use it myself, so that doesn't really matter.
I tried to replicate the configure line from my php4. Big mistake. All I got were segfaults and link errors. I re-configured with the minimalist line above. Now everything is happy.
Posted by billo at 06:24 PM | Comments (0)
March 31, 2006
NTS: doing a remote install of linux with VNC
Note to Self
This is about the coolest thing ever.
I would like to upgrade my dedicated server to COS4, and this just might be the ticket.
Yes, I am a geek.
Posted by billo at 01:55 PM | Comments (0)
March 17, 2006
Do not upgrade your Fedora Core 4 kernel to 2.6.15
I recently ran "yum update" on a bunch of machines at work. They were all "new" machines, and were not doing much anything interesting yet. Which is a good thing.
Apparently, somewhere between 2.6.11 and 2.6.15, some stuff with "udev" and partition labeling changed, because none of these systems would boot any longer. There were two problems: the first, which was that using "root=LABEL=/" in the grub boot loader no longer works. That was easy enough to fix by booting the old kernel and editing the grub.conf. The second problem is that the boot sequence needs /dev/console, which is no longer available early on in boot sequence on kernels that are "pure" udev. Whatever that means.
Ugh.
So my free advice is don't run "yum update." Instead, run "yum --exclude=kernel update"
Posted by billo at 03:21 PM | Comments (0)
March 16, 2006
SSH tunneling VNC
Sometimes I need to be able to get into my Mac at home, with the the full GUI and not just ssh. To do this, I use ssh tunneling. There are many variants of this recipe, but if your network is like mine, this one will work for you.
At home I have a fixed IP address, and I have my router set to forward port 22 to my bastion host. This is a minimal linux server with nothing much on it. To log in to any machine at home, I ssh to the fixed IP of my router, which forwards to the bastion host. Then from there I can ssh to any machine at home: the mac mini in the family room, the old g4 cube in the office or the old pentium 3 linux machine that serves as my internal DNS.
Caution: It's of course very important that the bastion host be kept up-to-date, and have very good passwords on the few account that are ssh-able. More than once I've had my house "hacked" because of vulnerabilities in various network services, and that is no fun. Fortunately, I've always detected the hackage within a few hours because of tripwires I had set up. It's also a very good idea to use IP tables to limit the places on the net that can connect to your SSH. For example, I only allow connections to my house from the IP addresses at work and at a couple of other places on the net. This makes my home server unreachable from the random hackers trolling for open SSH connections. This means, if I need to ssh tunnel from a new location (like from a Starbucks), I am out of luck.
On the Mac itself, you need to enable Apple Remote Desktop. This is nothing more than an enhanced VNC server. If you buy the ARD client it has some nice extra features like remote software updating, and probably optimized performance. But you can use any free VNC view to connect. I like Chicken of the VNC, mostly because it has a silly name. To enable ARD on your Mac, open System Preferences, click Sharing, check "Apple Remote Desktop," open the "Access Privileges" dialogue. Enable "VNC Viewers may control screen" and choose a password. The password could be something simple, because you aren't going to expose VNC to the network, you are tunneling over a secure connection.
For our example, let's pretend that your router fixed-IP has a DNS name of home.example.com; your mac at home has an internal DNS name of mini.home.example.com.
From outside your network, on your Mac with Chicken of the VNC, open a terminal and do:
ssh home.example.com -L 5901:mini.home.example.com:5900
After you sign in with ssh, your terminal window will show you signed in to the bastion host. You need to leave that running to keep the tunnel going.
Now, open Chicken of the VNC. You should add a new server entry, as shown. Use "hostname" and Display 1. Display 1 is 5901; if you tunneled from 5902, the display would be 2. The password should be the simple password you created on the remote Mac under Apple Remote Desktop access controls.
Click "connect" and you'll be connected. One more tip: don't put background images on your Mac desktop at home, since this will make everything REALLY slow. A solid background color is best.
Posted by billo at 08:30 AM | Comments (0)
March 13, 2006
Creating a kickstart install server for Fedora Core 4
We have a lot of servers at our engineering lab at work. It's not a lot by google standards, which is rumored to have at least 100,000 and maybe 1,000,000 servers worldwide. But we have something like 25 machines, and we have no dedicated sysadmins. Several of our programmers and release engineers sort of share the burden of setting up and deploying machines.
Back the the early days of our company, (in 2004), we had about 10 servers, and they were all the same hardware. Exactly the same. So we would build a master disk and clone it with "dd." That made creating machines very fast,
but it broke down as we collected more diverse hardware. And creating a master clone was a manual process.
"Kickstart" is a Red Hat feature that allows you to script the install procedure in the Red Hat system installed, anaconda. In theory, you should be able to put a new server on the network, do a "network boot" (PXE boot) to a master install server that had the linux images you care about (Fedora Core, in our case), and then run a scripted install without an administrator baby sitting the process. Then it is only a matter of assigning a real fixed IP address to that machine and putting it on the network.
It sounded really easy, so I figured I would read a man page, set up a bootp or tftp server, write the install script, and be done in no time. It took quite a bit longer than that, and was quite a bit more complicated. Here's what I learned.
Three Things: PXE, pxelinux, Kickstart
The first thing I learned was that there wasn't one thing that was the "kickstart network boot installation server." There are three components: PXE, which is a networking standard that allows a computer on a LAN to get an IP address, find out where some bootable images live, and start a boot. It could be linux, it could be any OS; pxelinux, which is a linux loader that can work within the PXE standard; and kickstart, which is the Red Hat / Fedora thing that enables doing an install from a distribution on a network (via FTP or HTTP).
I tried really hard to see if it was possible to do the last part without the first two, just because I am a lot more comfortable setting up an HTTP server and all that other stuff. But if there is a way to do it (and remain completely independent of having a boot CD-ROM or floppy), then I could not figure it out.
PXE
PXE (Preboot Execution Environment) is a standard driven by Intel and other vendors. You can read all about it on that Wikipedia link. The short of it is that it is tied in to DHCP. So you basically need to configure a DHCP server on your LAN so that machines can connect, get an IP address, and be told where to get a bootable image of some sort. Since there generally already is a DHCP server on one's network, and it generally is some kind of firewall/router/whatever (at least on small networks like ours), one doesn't really want to go mucking with it. The upshot of this is that one really needs to create a separate LAN, on a separate switch/hub that has its own DHCP server.
So I took our "master install" server, which has the FC4 distro on it, and two ethernet devices, and connected its second network port to an empty switch. I stuck a label on it that said "INSTALL NETWORK 192.168.0.0/24" so no one would plug anything else into it. [Our regular LAN at the office is 10.0.0.0/22.] Then I did some reading and searching, and cobbled together this dhcpd.conf, which causes the dhcp server on this master machine to only listen to requests on the install network, and not on the "real" network of the other card. It is very bad if you have two DHCP servers on the same LAN giving out addresses. Very bad.
# DHCP Server Configuration file.
ddns-update-style interim;
ignore client-updates;
local-address 192.168.0.1;
subnet 192.168.0.0 netmask 255.255.255.0 {
option routers 192.168.0.1;
option subnet-mask 255.255.255.0;
option nis-domain "example.com";
option domain-name "example.com";
option domain-name-servers 192.168.0.1;
option time-offset -18000; # Eastern Standard Time
option ntp-servers 192.168.0.1;
option netbios-name-servers 192.168.0.1;
option broadcast-address 192.168.1.255;
allow booting;
allow bootp;
range dynamic-bootp 192.168.0.128 192.168.0.199;
default-lease-time 3600;
max-lease-time 4800;
}
group {
next-server 192.168.0.1;
filename "pxelinux.0";
# only install one machine at a time.
# this is because the kickstart doesn't
# really work with dynamic ip, as far as i can tell
host install1 {
hardware ethernet 00:11:43:59:88:6C;
fixed-address 192.168.0.201;
option host-name "install1";
}
}
The key to this is is the "filename" directive which I highlighted above. This tells the booting machine (with the given MAC address) to ask the TFTP server at 192.168.0.1 for the file called pxelinux.0. This is the bootloader that you can then use to do different things: you could totally boot a Knoppix-style linux at this point. Or you could just run an installer, and that's what I wanted to do.
pxelinux
To get pxelinux going, you need to set up a TFTP server on your master install machine. The root of the tftp server should have an arrangement of files like this:
This is basically the default arrangement of pxelinux; pxelinux is a part of, or maybe a synonym for, "syslinux" which you can install on your server by doing "yum install syslinux" ; you can then copy most of the above hierarchy from /usr/share/system-config-netboot/ to your tftp root (by default that is /tftproot).
The interesting files in the above are:
- fc4: this is nothing more than a copy of the linux kernel from the FC4 install CD. It is found at ./isolinux/vmlinuz.
- fc4-initrd: this is a copy of the file ./isolinux/initrd.img from the FC4 install CD.
- memtest86: this is another kernel from the FC4 install CD. It's just a diagnostic image that will test RAM on your machine, and not harm anything. It's a good idea to make this the default boot image, when we get to that later. So you don't accidentally do a complete FC4 install on some random machine that accidentally is connected and netbooted to your install network.
- pxelinux.cfg/default: this is where most of the magic happens. This file is to declare the boot/install actions you want.
Here is the pxelinux.cfg default file I came up with. It's working for me, but I think there are still some things wrong with it.
# show the boot prompt, so the user can select "ks" or "memtest86" prompt 1 # make memtest be the default, so people don't erase their OS by accident default memtest86 # wait a long time before selecting timeout 10000 # harmless kernel label memtest86 kernel memtest86 # kickstart installer kernel label ks kernel fc4 append text initrd=fc4-initrd ramdisk_size=8192 ip=192.168.0.50 netmask=255.255.255.0 gateway=192.168.0.1 dns=192.168.0.1 ks=http://192.168.0.1/linux/isolinux/ks.cfg ksdevice=eth0
The interesting part of this file is the "ks" section. It's the part that points to the FC4 installer kernel, and the initial ramdisk image. Then it assigns a specific IP address for the installer to use, and specifies a kickstart file. This is where I have run into quite a bit of trouble. Supposedly, you can say "ip=dhcp" (see green highlight). But whenever I do this, the installer can't get it's DHCP address for some reason. It just hangs on the screen for selected IP address, and stays there until I put in a fixed IP. For now, I am living with this sub-optimal config, but it does work, and it is unattended, except when you start the install and when you finish.
Update: It actually turned out to be the managed switch I was using for my install network. There is a funny timing thing related to spanning tree that caused the DHCP requests from the install target to time out.
I just put in a cheap, unmanaged switch instead, and dhcp works fine now!
Kickstart
I had a very hard time finding any good, up-to-date, comprehensive documentation on Kickstart. The best thing I could find were older Red Hat documents, even though the feature is clearly in FC4. From this doc, and a nice article at fedora news, I was able to cobble together this script.
# Install a fresh system rather than upgrade an existing system install # Perform the kickstart installation in text mode text # Install from an installation tree on a remote server via FTP or HTTP url --url http://192.168.0.1/linux/ # Sets the language to use during installation lang en_US # Sets the language(s) to install on the system langsupport en_US # Sets system keyboard type keyboard us # auto mouse mouse # Sets up the authentication options for the system authconfig --enableshadow --enablemd5 # Sets the system time zone timezone America/New_York # Configures network information for the system #network --device eth0 --bootproto bootp # doesn't seem to work #network --device eth0 --bootproto dhcp # doesn't seem to work network --device eth0 --bootproto static --ip=192.168.0.51 --netmask=255.255.255.0 -\ -gateway=192.168.0.1 --nameserver=192.168.0.1 # Sets the system's root password to "whatever" rootpw whatever # no firewall firewall --disabled # No SE Linux! selinux --disabled # Specifies how the GRUB boot loader should be installed bootloader --location mbr --append quiet # Removes partitions from the system, prior to creation of new partitions clearpart --all --initlabel part swap --recommended part /boot --fstype ext3 --size 100 part / --fstype ext3 --size 1024 --grow # Package Selection %packages @ base # Post-installation Script %post rpm --import /usr/share/rhn/RPM-GPG-KEY-fedora # magic config wget http://192.168.0.1/kickstart/post_install.sh sh post_install.sh
The main point of the above is to answer all questions the installer would ask interactively, and install the bare minimum package ("base"). Then, the magic escape sequence is the last two lines of the file: we get a shell script from our master server, which is called post_install.sh. This does anything we want, but mainly:
- it installs all individual packages we want, and does it from a local yum server so it is fast.
- fetches and installs any extra things we want, like java or our own scripts.
- creates basic user accounts and groups we want, and points to our LDAP server.
- anything else you can write a script for
That's all there is. I would like to figure out the DHCP thing, so I could boot/install 4 or 5 (or 50) machines in parallel. But for now, this is pretty good.
Posted by billo at 03:44 PM | Comments (4)
March 09, 2006
Make your own yum repository
If you are a Linux Fedora Core user, you might be familiar with the package management tool "yum." Yum is nice, and similar to pacman (on arch) and apt (on debian/ubuntu). It's a little bit annoying the way it has a huge (25-second) startup while it reads in and parse meta-data. But otherwise it works well.
If you have a lot of machines to keep up to date, however, it is inefficient to have all of them phone home to the mirror sites out on the net to update packages. At least it is when you are on a wimpy T1. I found this document on creating your own local yum repository. It is very succinct. I also found this one, which is way more wordy than necessary, but it gives more background, and includes info on seeding the base yum repository from the FC4 distro CDs.
The bad part of it is that you have to pay the one-time bandwidth pain of getting the entire FC4 updates repo mirrored to your own server. It's pretty huge, at least 2GB.
For my own reference, this is what I am doing:
rsync -av --exclude debug --exclude=repodata/
'--exclude=*debuginfo*' '--exclude=*i18*'
'--exclude=*langpack*'
'rsync://mirrors.kernel.org/fedora/core/updates/4/i386/*'
/var/ftp/pub/yum/4/i386/updates/
createrepo -q /var/ftp/pub/yum/4/i386/updates/
repoview -q /var/www/html/yum/4/i386/updates/
Posted by billo at 04:18 PM | Comments (0)
March 02, 2006
Software Update Heart Attack
I installed the latest security patch from Apple this morning. It required reboot (come on, Apple, quit it).
After reboot I could not log in with my normal username. We're using an LDAP login on our network, so I figured it was some kind of hiccup with that. Not to worry, I thought, I'll just use my backup (local) user account and see what's what.
Oops, that one wasn't working either. Oh crap. Did I forget the password?
Panic starting to set in. I have a lot of work to do today, and I don't want to blow the morning doing an OS repair. And I certainly don't want to endure the taunts of the Windows who sits at the desk on my left. Or the Linux user who sits on my right, for that matter. No problem, I'll ssh in using my public key. That will bypass any password issues, LDAP or otherwise.
So I hopped on to my backup keyboard (connected to my backup machine, which happens to be Ubuntu Linux).
slayer% ssh buffy ssh: connect to host buffy port 22: Connection refused slayer%
OK. Now I'm freaking out. I really don't want to pull out the install DVD. In desperation, I reboot again. That should make no difference, right? Wrong. Everything is fine on reboot. As if nothing was ever wrong. No trace of a problem can be found in the system logs. Ugh.
Lessons:
1. make sure you know your backup account password.
2. don't accept reboot patches in the middle of the morning.
3. think seriously about switching to linux full time. (giving up itunes would be hard)
UPDATE: Derek just did this update on his mac, and it actually had a thing that said "ok you need to reboot one more time." Looks like they patched the patch, and I am not nuts.
Posted by billo at 01:52 PM | Comments (0)
February 08, 2006
Linux Machine as File Server for Macs
I'm not sure exactly why I want to try this, but it seems like a good idea. I have this super cheap Dell file server I built for less $1/GB. It has 1TB of disk storage (on 4 250GB disks), and was about $900, including the server itself. It's some low-end model that has 4 SATA slots. The key is not buying the disks from Dell. You can get 250GB SATA disks for about $110 now, and the server is like $399 or something. I'll bet you could get one even cheaper if you tried.
In any case, the only way I have been using the server is do network backups of my Macs at home: an new one (mac mini), a sort of old one (2003 G5) and a wicked old one (2000 G4 Cube). Each mac is pretty isolated otherwise, and there is a lot of overlap, and the small one (the mini) is always low on disk space. So I wanted to be able to use that big file server to do something like store all my music files and iMovie projects in one place (not counting backups). But I've had bad luck in the past mounting network file systems on a Mac (samba or NFS). It seems like the lack of resource-fork stuff almost works, but not quite. Things get weird, and break in odd ways.
So my hope was that I could have the Linux machine serve up Apple shares more "natively" and that would make things better. I did some research and found netatalk, which is a full implementation of an Appletalk file server for unix. So far, it's worked like a charm. I installed the RPM, and was able to connect to my home directory on the linux machine without any problem. I'll have to try more things later tonight, like editing an iMovie project over the network and see how that goes.
Posted by billo at 02:33 PM | Comments (0)
November 22, 2005
Vuescan is AWESOME.
I have this cheap Canon USB flatbed scanner that I bought about a year ago. I use to have a fancier HP scanner that I bought about 8 years ago, but it is SCSI, and I don't really have any computers that do SCSI any more. (It's really annoying that there is no Mac OS X support for the Adaptec 2940 SCSI PCI card that I have, but that's a rant for another day.)
When I got the scanner, I had a hard time getting it working. Mostly this is because (unsurprisingly, I guess) the drivers that came with it were complete garbage. When I got it, the drivers were for the Mac OS version that was already old, and they haven't updated their downloads on their web site.
In any case, I needed to use the scanner the other day, and I realized I had reinstalled my OS since the last time I used it. And whatever hacks I did to get the scanner to work before, I didn't remember. Or maybe they just wouldn't work with Tiger.
After some googling around, I found Vuecan, a third-party scanner software package that claims to support most scanners. I tried the free download, and it worked perfectly. No setup, just copy the app into place, and it works. It's a really nicely usable package too; it strikes the balance well between being a data capture tool, and a useful imaging application.
They have a Linux version also, so I am going to try to revive my old SCSI scanner on my Ubuntu Linux machine.
So, if you're on a Mac, need to scan, Vuescan is well worth the $49. Just toss those junky OEM drivers in the trash.
Posted by billo at 09:34 AM | Comments (0)
July 11, 2005
Setting up a reliable server
Usually when I set up a new linux computer system, my goal is to make the cheapest thing possible, and set it up as simply as it can be. The main idea is that if the hard dies or something, it's easy to just replace the dead machine with an identical clone in a short period of time. Of course, administrative scripts, recipes, and BACKUPS are key.
I am starting to put together a server for my wife's business. For various reasons, the cheap throwaway strategy won't work in this case. Part of it is the business cost of any downtime, part of it is the fact that there is no on-site IT staff (i.e. me).
So I'm actually putting together a highly available, reliable server. I'll document how it comes together here.
For starters, I bought a semi-fancy Dell 2800 series tower server. It has dual CPUs, dual power supplies, dual network interfaces, 2GB RAM. It has a split SCSI backplane with hardware RAID 1. I've had it configured so there are three logical drives, each of which are made out of two physical disks.
Logical disk 0 is the operating system. I'm starting with Arch Linux, which is a nice distribution that uses a debian-style package management system called "pacman". It's nice because it's all command-line based.
Logical disk 1 is the application software and database disk pair. It's 75 GB logical disk, which should be 10X what we actually need.
Logical disk 2 is a 300GB storage unit. The application we will be installing has document management and imaging capabilities, and this is where that data will be kept.
Finally, I have a dual backup strategy planned. First, there is a big, fancy, painfully expensive digital tape system that as a 200GB capacity. Then, I have a cheapo server that I am building, which is a $399 low-end dell server with 4 big SATA disks. This will give me over 1TB of network disk for over-the-network backups at a total cost of less than $800.
Posted by billo at 11:43 AM | Comments (0) | TrackBack
May 18, 2005
Regarding server deployments
Joel on Software has a nice learning document about setting up a small multi-machine web hosting environment. Almost everything he says corresponds to my own experience.
I'm most interested in the article for his comments on the Dell DRAC4 remote management feature. We have to make a decision on that for our data center at kayak.com. Right now we have Raritan IP KVMs, and since we are about to move from 45 up to 90 machines or so, avoiding additional raritan expense would be nice.
Posted by billo at 09:27 PM | Comments (0)