macronet

Downtime 16.7.2014

 Maintenance  No Responses »
Jul 172014
 

Hi,

Couple of servers got broken about simultaneously, everything moved to a new platform without any dataloss.

Sorry for a short downtime, keep calm and carry on!

Resizing an encrypted RAID (and converting RAID5 to RAID6)

 Kernel and drivers  No Responses »
Mar 232014
 

Remember: everything you do, happens on your own risk. Steps below CAN CAUSE DATA LOSS. Do not run in production without understanding what you are doing and testing it in development environment beforehand.
And don’t blame me if it doesn’t work for you.

Before: 5 disks in RAID5 with LUKS -encryption and LVM
After: 7 disks in RAID6 with LUKS -encryption and LVM

RAID details: # mdadm –detail /dev/md0

  1. Add spare drive to RAID5-array: 
    # mdadm --add /dev/md0 /dev/sdf
  2. Convert RAID5 to RAID6: 
    # mdadm --grow /dev/md0 --level=6 --raid-devices=6 --backup-file=/root/raidbackup
  3. WAIT for the array to rebuild itself (will take a while): 
    # cat /prod/mdstat
  4. After rebuild, add another drive to RAID6-array: 
    # mdadm --add /dev/md0 /dev/sdg
  5. Take the drive into use (adding makes it a spare): 
    # mdadm --grow /dev/md0 --level=6 --raid-devices=7 --backup-file=/root/raidbackup
  6. WAIT for the array to rebuild itself (will take a while): 
    # cat /proc/mdstat
  7. Grow the RAID6-array to the maximum size: 
    # mdadm --grow /dev/md0 --size=max
  8. Open encrypted disk (if not already open): 
    # cryptsetup luksOpen /dev/md0 crypt
  9. Resize encrypted disk: 
    # cryptsetup resize /dev/mapper/crypt
  10. Resize LVM physical volume: 
    # pvresize /dev/mapper/crypt

And now you have more space what you can manage with LVM.

Note: If i remember correctly, 7. can be combined with 5.:

  • # mdadm --grow /dev/md0 --level=6 --raid-devices=7 --size=max --backup-file=/root/raidbackup

– and resizing could be done while the array is rebuilding, but just to be sure we are on the safe side…

iSCSI broken on Solaris 11.1

 Kernel and drivers, Solaris  2 Responses »
Jan 232013
 

After updating a Solaris 11.0 installation to Solaris 11.1 with pkg update – none of servers iSCSI -targets were accessible. Under Windows disk management partition would show up as RAW -partition, data could still be rescued with recovery -software.

After troubleshooting it came clear that i’m not the only one.

Problem is in ImmediateData, and it seems to be resolved in the latest Solaris 11.1 -update (0.175.1.3.0.4.0) – which requires a support contract with Oracle to be downloaded….

 

If you don’t have access to Oracle Solaris Support -repository – stay in 11.0 until next major update is released, or use a workaround which disables ImmediateData:

Windows (tested):
1. Set the following registry value to 0:
HKLM\SYSTEM\CurrentControlSet\Control\Class{4D36E97B-E325-11CE-BFC1-08002BE10318}\<Instance Number>\Parameters\ImmediateData
2. Reboot

Linux (tested):
1. Disable ImmediateData in iscsid.conf (location depends on distro) or use the same iscsiadm -command as in Solaris
2. Restart the iSCSI service

Solaris (untested):
1. Disable ImmediateData:
# iscsiadm -m node -T ${IQN} -p ${IP}:${PORT},${TPGT} -o update -n node.session.iscsi.ImmediateData -v No
2. Restart the iSCSI service:
# svcadm restart svc:/network/iscsi/initiator:default

ESXi (untested):
1. Backup vmkiscsid.db
2. Edit vmkiscsid.db with sqlite3:
select * from nodes; update nodes set ‘<immediatedatakey>’ = ‘No’
3. Replace vmkiscsid.db with modified version (if you didn’t edit it in place) and reboot server

Upgrading from Debian Squeeze (6.0) to Debian Wheezy (7.0)

 Legacy, Linux  No Responses »
Jan 202013
 

Debian Squeeze was released 06.02.2011.
Please note that Wheezy is still in testing -stage.  Debian Wheezy was released 04.05.2013

Just some quick steps how to do the upgrade (on your own risk).

Update Squeeze

aptitude update
aptidude upgrade
  1. Copy /etc/apt/sources.list to /etc/apt/sources.list.d/debian-wheezy.list and replace squeeze with wheezy. Or copy them inside sources.list. Use whatever mirror which is closest to you. 
    deb http://ftp.fi.debian.org/debian/ wheezy main contrib non-free
deb http://ftp.fi.debian.org/debian/ wheezy-updates main contrib non-free
deb http://security.debian.org/ wheezy/updates main contrib non-free
  2. Update repository 
    aptitude update
  3. Upgrade critical parts first – it will complain about libept1 – just let it be removed. 
    aptitude install dpkg apt aptitude
  4. (Dist-)upgrade rest 
    aptitude upgrade
aptitude dist-upgrade
  5.  IF YOU ARE STILL RUNNING UNDER XEN3.x:
    Replace grub2 with grub1 (or just keep your old menu.lst at /boot/grub/)

    aptitude purge grub-pc
aptitude install grub-legacy
  6. Always check that /boot/grub/menu.lst or /boot/grub/grub.cfg exists and defaults to right kernel
  7. Reboot and hope for the best

 

<complete instructions>
https://www.debian.org/releases/stable/i386/release-notes/ch-upgrading.html
</complete instructions>

Debian 7.0 desktop with i845G/GL

 Kernel and drivers  No Responses »
Jan 202013
 

Debian 7.0(beta4) and i845G/GL didn’t work out-of-the box for me, X crashed without a log as soon as it tried to initialize.

After adding following to /usr/share/X11/xorg.conf.d/05-i845g.conf – everything seems to work:

<code>

Section “Device”
Option    “DRI”    “True”
Option    “Shadow”    “True”
Option    “XvMC”    “False”
Option    “XvPreferOverlay”    “False”
Identifier    “Card0”
Driver    “intel”
VendorName    “Intel Corporation”
BoardName    “82845G/GL [Brookdale-G]/GE Chipset Integrated Graphics Device (rev 01)
BusID    “PCI:0:2:0”
EndSection

</code>

It’s possible that this is fixed when Wheezy is officially released, at least a bug has been reported.

 

Linux 2.6.37 kernel and rr268x driver (v1.6)

 Kernel and drivers  6 Responses »
Mar 132012
 

After quick hack-n-slash, it works. It works also on 3.0, if you want 3.1/3.2/3.3 – modify it yourself or wait until i need it 🙂

Needs quite non-intrusive modification to three files:

  • rr268x-linux-src-v1.6-legacy_single/osm/linux/osm_linux.h
    — fix autoconf
  • rr268x-linux-src-v1.6-legacy_single/osm/linux/os_linux.c
    — in 2.6.36 blkdev_get requires two parameters, in 2.6.37 three
  • rr268x-linux-src-v1.6-legacy_single/inc/linux/Makefile.def
    — mask 3.0 as 2.6, can be used to mask 3.1/3.2/3.3 also, and could probably be a lot prettier

 

Patchfile can be found here: http://macronet.fi/dev/rr268x/ < not recommended, get official fixed version from highpoint (unless you are absolutely sure you want to run v1.4 or v1.6).

* Update 05/2012 *  v1.6 patchfile found in /staging which is modified to mask all 3.x as 2.6

* Update 07/2012 * Just get v1.8 driver which is properly fixed by Highpoint:
http://www.highpoint-tech.com/USA_new/rr2600_download.htm

* Update 01/2013 * There seems to be v1.9 driver released, with following info in README:
NOTE: The latest tested kernel version: 3.5.2.
v1.9.12.0817 08/17/2012
* Fixed a potential bug about fail to recover array.

/64 IPv6 -network to LAN: bridge

 Legacy, Network  No Responses »
Jan 182012
 

One way to get IPv6 /64 bridged to your LAN:


brctl addbr br0
brctl addif br0 eth0
brctl addif br0 eth1
ebtables -t broute -A BROUTING -p ! ipv6 -j DROP
ifconfig br0 up

 

Opened up: create software bridge from your WAN -interface to LAN -interface, but allow only IPv6 to fly over it. In my setup IPv4 is natted.

IPv6-trouble with fully virtualized Debian

 Linux, Virtualization  No Responses »
Dec 102011
 

Once upon a time, virtual was installed under full virtualization (KVM) -mode. Network wasn’t bridged, it was routed.

And then the problem: IPv6 -traffic flowed nicely inbound, but outbound was capped to about 128 kbps.

Troubleshooting: after googling around with ipv6, kvm and debian we came around to this, Debian bug report about GSO Ipv6 issues under KVM. It’s supposed to be fixed in 2.6.32-5-amd64 (2.6.32-39) -package…well, upgrading didn’t work for us, propably another (similar kind of) bug or something needs to be done at host-machine.

Workaround:  disabling virtio_net’s gso worked for us:

ifdown eth0; modprobe -r virtio_net; modprobe virtio_net gso=0; ifup eth0

Upgrading from Debian Lenny (5.0) to Debian Squeeze (6.0) under Xen

 Legacy, Linux, Virtualization  No Responses »
Dec 092011
 

Debian Squeeze was released 06.02.2011 and Lenny’s support will (probably) be is discontinued since 06.02.2012.

 

Problem: apt-get update && apt-get dist-upgrade broke everything, server did not boot anymore.

Troubleshooting: squeeze has grub2, which our Xen 3.x and pygrub didn’t understand

Solution: upgrading with following steps, use apt-get or aptitude, whichever you like.

  1. Update Lenny 
    aptitude update
aptidude upgrade
  2. Replace lenny with squeeze in /etc/apt/sources.list – or add necessary entries to another file under /etc/apt/sources.list.d/ 
    deb http://ftp.fi.debian.org/debian/ squeeze main contrib non-free
deb http://ftp.fi.debian.org/debian/ squeeze-updates main contrib non-free
deb http://security.debian.org/ squeeze/updates main contrib non-free
  3. Update repository 
    aptitude update
  4. Upgrade critical parts first 
    aptitude install dpkg apt aptitude
  5. (Dist-)upgrade rest 
    aptitude upgrade
aptitude dist-upgrade
  6. Replace grub2 with grub1 (or just keep your old menu.lst at /boot/grub/) 
    aptitude purge grub-pc
aptitude install grub-legacy
  7. Check that /boot/grub/menu.lst exists and defaults to right kernel
  8. Reboot and hope for the best

 

Also remember to use 2.6.39 -kernel from backports for live migration to work.
Update 01/2013: 2.6.32 -kernel usually works just fine, Lenny’s EOL date is 6.2.2012.

 Newer Entries