You are currently browsing the category archive for the ‘Uncategorized’ category.

**** Prepare the media installer

as the lian Li PC-Q25 has no space for DVD, it require to burn  FreeBSD usb stick image

so, lets go to http://www.freebsd.org/where.html

I want 9.1 release as it’s tthe latest production release when I did my install

basically, unless you have a very old processor ( let say more than 5 years), choose always AMD64 even if you got an INTEL processor.

so I get :ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/9.1/FreeBSD-9.1-RELEASE-amd64-memstick.img

then if you have a windows box, download winimage at https://launchpad.net/win32-image-writer

of course at this step, you need a usb key fully free of a least 1 GB to burn the downloaded image to it

it’s easy to use…before launching it, unplug any usb disks/keys to avoid write on the wrong device: )

then plug only the usb stick that will be used  to write the freebsd installer image.

start winimage.Exe, choose the downloaded image, select the right device, click write, wait some minutes…and you have a bootable freebsd usb key installer .

*** BIOS configuration

plug the usb stick

boot and enter bios

During the assembly, I put a stick “DISKxx” on each disk , noticed the serial number, and I also noticed to which SATA ports which disk is connected

so I have :

PORT BIOS -> PORT MOTHERBOARD ->PORT CASE ->HARD DISK-> SERIAL NUMBER
P1 -> SATA 6G_1  -> CON2(top)  ->
P2 -> SATA 6G_2  -> CON2(bottom)
P3 -> SATA 3G_1  -> bottom right ->
P4 -> SATA 3G_2  -> CON1(bottom)   ->  DISK01 -> WD-WMC1T1xxx663
P5 -> SATA 3G_3  -> CON1(bottom)  -> DISK02 -> WD-WMC1T0xxx491
P6 -> SATA 3G_4  -> CON3(top)  -> DISK03 -> WD-WMC1T0xxx123

this will help to build the system and in case of hard drive failure

so I setup my bios to boot disk order P4,P5,P6

I also setup SATA mode as AHCI and Hotplug Enable for all drive

reboot and press F8 (my bios key to select boot device), I select the usb key

if things are ok … you should freebsd booting and then you will have :

installer1

select “Shell” n then you can enter commands ….

for french users, type :

kbdcontrol -l fr.iso

-> to have a french keyboard 🙂

for  other language , find your.. :

ls -1 /usr/share/syscons/keymaps/ | more

just don’t put the .kbd in the -l argument…

example  kbdcontrol -l danish.iso for danish

as I want to do the install not through local console, but from the network with ssh client (this will help to copy/paste,etc..), this would be insecure as the ssh root access will be temporary without password ..but I trust my local network at home 🙂

theses steps are optional , theses won’t have any impact for the installed systems, use local console if you want , so you can go to step //END OPTIONAL if you do’nt want this extra step

// BEGIN OPTIONAL

to find the network interface, you can do

dmesg|grep  address

find_network_itf

so it’s em0,  below, replace em0 by the one you have

/sbin/ifconfig em0 up

you probably have a DHCP service on the network, so let’s use it to give an IP.. :

/sbin/dhclient  em0

installer_dhcp

to be able to reach the install through ssh :

ssh-keygen -t rsa -f /tmp/ssh_host_rsa_key

when it asks for passhprase, simply press ENTER

/usr/sbin/sshd -h /tmp/ssh_host_rsa_key -h /tmp/ssh_host_dsa_key -o PermitEmptyPasswords=yes  \

-o PermitRootLogin=yes -o usePAM=no -o  PasswordAuthentication=yes -f /etc/ssh/sshd_config

now, you can access it  by an ssh client..if you don’t recall the IP assigned to the installer..

/sbin/ifconfig em0

ifconfig_installer

notice IP adress after IP, for me it’s 192.168.193.128, below, replace this by the one you have !

now for another box, you can ssh this install

you can use putty :

putty_installer

feed the right IP, ensure Connection type is SSH and so Port is 22, then click Open

when you see “login as:” simply enter root

//END OPTIONAL

type :

kldload opensolaris
kldload geom_mirror

this will load kernel modules to deal with our zfs storage and  swap mirroring

now it’s time to identify our disks in the systems…I assume you use only SATA disks!!

dmesg |grep ATA-

disk_list

depending on your number of hard drive you put, you may see ada0 to ada(N-1) where N is number of disks

be carefull, if you do’nt see all your hard drives, it means one or more can be dead or badly cabled…

check the serial number of your drive…

for all you adaX, type

camcontrol identify ada0 | grep serial

camcontrol identify ada1 | grep serial

so as previously, you put a stick on each disk ( DISKXX) and keep trace of the serial

you can now say ( it’s an example):

DISK01 is at ada1

DISK02 is at ada2

etc… but you could also have DISK01 at ada2 it depend on which SATA ports your disks are connected…

just make a note on a paper carefully of this mapping !!!as if you make a mistake at the previous,next steps  you will encounter big issues if you have a disk failure…as we will now use the information DISK0X  to “label” the disks from an OS point of view.

to sum up my idea

— 3 disks  use:

on each disk we will create a partition for hosting the system with zfs mirror

on each disk we will create anoher  partition for hosting the datas  with RAIDZ1

below I will only do for 3 diks, as it’s my current setup ( I’m currenty going to 6 disks for having RAIDZ2 and 2 disks lots without losing datas)

gpart create -s gpt da0
gpart create -s gpt da1
gpart create -s gpt da2

check that your disks is 4K sector :

camcontrol identify ada0 | grep sector

4Ksector

as you see for sector size, physical is showing 4096, not 512 , so my WD are 4K sector aligned…

to be able to setup partition with gpt tools for a 4K disks aligned ( releveant parameter -a 4096,  use -a 512 if disks is REAL 512  )

gpart add -b 2048 -a 4096 -s 64k -t freebsd-boot da0
gpart add -b 2048 -a 4096 -s 64k -t freebsd-boot da1
gpart add -b 2048 -a 4096 -s 64k -t freebsd-boot da2

this willl create necessary partition to host boot code

let assume DISK01 is at da0, DISK02 is at da1, DISK03 is at da2, if it’s not your case,  repalce your correct mapping below !

gpart add -a 4096 -s 2G -t freebsd-swap -l swap1-disk01 da0
gpart add -a 4096 -s 2G -t freebsd-swap -l swap1-disk02 da1
gpart add -a 4096 -s 2G -t freebsd-swap -l swap1-disk03 da2

this will be use to create after a 3 way mirror of swap of 2GB, if you think you need to have 4 Gb of swap , replace  -s 2G  by -s 4G  on each line

note: that even you don’t plan to use swap , because you have lots of memory, you still can create thes partitions .. and not use them …  what is 2, or 4 GB on a 3TB disk  ( 2,7TB in fact for the WD red 3TB) ?:)

gpart add -a 4096 -s 64G -t freebsd-zfs -l zROOT1-disk01 da0
gpart add -a 4096 -s 64G -t freebsd-zfs -l zROOT1-disk02 da1
gpart add -a 4096 -s 64G -t freebsd-zfs -l zROOT1-disk03 da2

this will be used to create a zpool of 64 GB (  this will be 3 way mirror!) , if you think you need less space for your system, not the datas! (  change it on each line)

but I allocate lots of space for OS ,as thanks to ZFS features ( snaphost, clones, receive/send) I will be able to install a fresh system without removing the current installed,  upgrade the onr already installed, be able to rollback the change is it’s bad,etc…

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da2

now  all partitions for systems are done, let create the zfs pool  that will host the system

to be sure zfs will take care of 4K aligned sectors :

gnop create -S 4096 gpt/zROOT1-disk01
gnop create -S 4096 gpt/zROOT1-disk02
gnop create -S 4096 gpt/zROOT1-disk03

if you have a 512 disks, you should use -S 512

zpool create -R /tmp/zroot  zROOT1  mirror gpt/zROOT1-disk01.nop gpt/zROOT1-disk02.nop gpt/zROOT1-disk03.nop

the pool will be named zROOT1 that why it was part of partition label of it’s members…:)

the system will be installed in a zfs dataset, so wy will be able to have multiples system, do so update, snashot the OS,etc…

to better understand what you could do with ZFS, you may check  this link

zfs create zROOT1/ROOT9.1

this will create a dataset ROOT9.1 , this will contains at our system

if you follow correctly my settings, you could in some month install for example a fresh system ,let says a 9.2 without deleting or interfering with the one we currently setup…

cd /tmp/zroot/ROOT9.1
mkdir -p boot/zfs
zpool set cachefile=” zROOT1

cd /tmp/zroot/ROOT9.1
tar -Uxpf /usr/freebsd-dist/base.txz
tar -Uxpf /usr/freebsd-dist/lib32.txz
tar -Uxpf /usr/freebsd-dist/kernel.txz
tar -Uxpf /usr/freebsd-dist/ports.txz

edit /tmp/zroot/ROOT9.1/boot/loader.conf
vfs.root.mountfrom=”zfs:zROOT1/ROOT9.1″
zfs_load=”YES”
opensolaris_load=”YES”
geom_mirror_load=”YES”

edit /tmp/zroot/ROOT9.1/etc/rc.conf
hostname=”NEW-NAS”
ifconfig_em0=”dhcp”
zfs_enable=”YES”
sshd_enable=”YES”
keymap=”fr.iso.acc”

of course, if you are not french, change keymap by the correct one for you country

you can do  an  ls -1 /usr/share/syscons/keymaps/ | more to find the one best for you, just don’t put the .kbd in the keymap directive…

also  correct the hostname and change ifconfig_em0 by ifconfig_XX  where XX is your network interface,  example : re0

if you want static IP :

syntax is like this one :         ifconfig_em0=”inet 172.31.1.200 netmask 255.255.255.0″     replace IP & netmask by your..

in this case, also add in defaultrouter=”172.31.1.254″  of course, put your router’s IP …

if you want the swap  :

gmirror label -b load -F swap1 gpt/swap1-disk01 gpt/swap1-disk02 gpt/swap1-disk03

edit /tmp/zroot/ROOT9.1/etc/fstab :

# Device         Mountpoint      FStype  Options Dump    Pass# 

/dev/mirror/swap1 none            swap    sw      0       0

below,

change NEW-NAS by the name you put in the rc.conf file

after “password root” , you need to enter the password you want for root

with “pw useradd admin -m -G wheel”  it create an account that you will use for ssh access, you will also need to enter a password

you should  change admin by something more customized, like your first name :), or you could create other account later

chroot /tmp/zroot/ROOT9.1
passwd root
pw useradd admin -m -G wheel
hostname NEW-NAS
tzsetup
cd /etc/mail
make aliases
exit

at tzsetup, choose the correct timezone…

cd /zpool
export zROOT1
gnop destroy gpt/zROOT1-disk01.nop
gnop destroy gpt/zROOT1-disk02.nop
gnop destroy gpt/zROOT1-disk03.nop

zpool import -o altroot=/tmp/zroot -o cachefile=/tmp/zpool.cache zROOT1
cp /tmp/zpool.cache  /tmp/zroot/ROOT9.1/boot/zfs

zfs set mountpoint=legacy zROOT1
zpool set bootfs=zROOT1/ROOT9.1  zROOT1
reboot

so before the reboot, we setup the pool zROOT1 to not be mount automatically  and setup  the boot filesystem for pool zROOT1 as zROOT1/ROOT9.1, so

at boot, freebsd bootcode will look at zROOT1/ROOT9.1/boot/loader.conf to know what will be as /
vfs.root.mountfrom=”zfs:zROOT1/ROOT9.1″
so if we do another fresh install in new dataset zROOT1/ROOT-test, we will have to change

zpool set bootfs=zROOT1/ROOT-test  zROOT1

and setup zROOT1/ROOT-test/boot/loader.conf to contains : vfs.root.mountfrom=”zfs:zROOT1/ROOT-test”

this seems a little complicated but you can try with virtualbox/vmware workstation/ virtual pc/ …

after reboot, remove usb stick and keep in save place, it could be usefull for emergency 🙂

Advertisements

CHASSIS

I would like to have a small chassis, ideally a mini-itx box
with at least 6 disks slot..why 6 , because with 6 disks I can have lot’s of solution for disks :

– for begin a RAIDZ1 pool of 3 disks , so I can loose a drive.. and if need more in the future, add a second RAIDZ1 pool with 3 disks
– a RAIDZ1 of 6 disks, so I can loose a drive and have around 5* size of the hard disk for the storage
– a RAIDZ2 of 6 disks, so I can loose 2 drives and have around 4* size of the hard disk for the storage

it need also “my wife agreement” for the design…
==> I found the Lian LI PC-Q25
I hesitate with the Fractal design Node 304
which provide usb in front but I wanted to look for an aluminium case…:)

DISKS

well… I search for big size, low noise, decent speed, be able to run 24/7.
so in 2012, western digital push RED disks, which are targetted for NAS use..
so let’s use their 3 TB model
you can see this review

let’s start with 3 to build a RAIDZ1 pool…

MOTHERBOARD

mini-itx with 6 builtin 6 SATA ports is not widespread..
to my knowledge the only model in 2012 was the ASUS P8H77-I
ASUS is a good brand, and appart the SATA ports, this model provide some virtualization feature so let take it.

CPU

My server is for home use, so few users at same time for file share, I target either an Intel Celeron G860 or an Intel I3-3220, because of TDP of 55W, both provide Intel vt-x technology for hardware virtualization and have GPU integrated…
I don’t want a processor with higher TDP because mini-itx chassis does not offer lots of room for cooling
and a quad core should be too much for my use.
Overclocking is not a requirement for me…it’s power consuming for few more horse power…

I choose the I3, it’s more expensive but have more speed, I will drink less for some time to pay the gap 🙂

RAM

I don’t need overcloked kit, ZFS like RAM and even if 4GB should be enough , I decide to buy a 2x 8GB kit , so I can deal with some virtualization without changing the hardware..
RAM is not expensive now…

POWER SUPPLY

without a dedicated graphic card and a non overcloked low TDP CPU, with 6 disks, I expect my sytem to be largely under 150W on big load…, so I would need a psu proviing between 200 and 300W max…
I choose this : Seasonic G-Series G-360 – 360W
it seems stable, reliable for 24/7…and very quiet
Be carefull.. there are only 4 SATA and 3 MOLEX connectors…so as the LIAN LI PC-Q25 have hotplug backplane for 5 disks with 3 MOLEX, all MOLEX are used!

CPU COOLER (OPTIONAL)

I don’t like intel stock cpu cooler…lots of noise and few efficiency. Due to mini-itx motherboard & chassis compatibility , I take this one Noctua NH-L9i
Noctua is known for good reliability, low noise and efficiency.

To sum up :

  • Asus P8H77-I
  • Seasonic G-Series G-360 – 360W
  • 3 x Western Digital WD Red RD1000M – 3 TB
  • Lian Li PC-Q25
  • G.Skill Kit Extreme3 2 x 8 Go PC10600 ARES CAS9
  • Noctua NH-L9i
  • Intel I3-3220

PRICE   when I bough it on 30/11/2012 : 953€..

you could save money, probably  around 100€  by buying only 4GB RAM, using intel stock cooler,and using an Celeron processor. note that my target is not to build the cheapest solutionn it’s a balance with  cost, quality. I have some money to spend 🙂

you may say… where the OS will be installed…

short answer : on the disks assigned for the storage…  ther are more advantages rather than using an usb stick…:)

I need to have theses features :
– network sharing for documents, music, phots and videos
– music stream server, accessible from internet with web browser and smartphone
– download server, something like synology download station
– ftp server, accessible from internet
– web server with php, mysql for possible small web hosting
– possibility of virtualization
– ability to sync datas with other server/NAS
– disks fault tolerance
– disks monitoring
– filesystem snapshots in case of human error file deletion

so good candidates are freenas,nas4free, or go to linux distribution or freebsd release.
I personnally exclude openindiana, and other things based on solaris or opensolaris because I’m more experienced on linux/*BSD and supported hardware is more limited
Also, I exclude microsoft OS (windows server/home server) due to extra cost and feature limited for my need

I decided to go to use FreeBSD 9.1 for OS… I want ZFS and fully setup system like I want

Next step is hardware buy…