Monday, April 6, 2009

Creation and Maintenance of a Gentoo Backup

This guide will instruct you in create a tarball with a up to date Gentoo system which can be used to save much time when formatting and reinstalling.
First of all, I will call those tarballs as follows (from here on):

1. stage4 -> A complete bootable system with kernel, grub but without Xorg, Desktop environment nor anything Xorg related.
2. stage5 -> It will contain all from stage4 and it includes too Xorg, Desktop Environment and every application you use.


Starting installing Gentoo
First of all, you need to install gentoo from scratch following Gentoo's Handbooks until the "Configuring Boot Loader" but without reboot nor unmount nor exitting chroot at the end.

Updating Gentoo
Now it's time to update the base system of gentoo by running: emerge --sync && emerge -uDN world
After update, you can proceed with installation of any non graphical application and config that will be saved: For example, emerge networkmanager and executing rc-update add NetworkManager default

Compress the stage4
Now it's time to compress the stage4 image to be used in case we need to format and reinstall.
If your idea is to have stage5 instead of stage4, just skip this section, and continue to Install Xorg and Desktop Environment.
Now type exit in order to exit from chroot, and mount the destination filesystem to store the tarball but keep in mind that it will need to be stored in a partition different than where you installed Gentoo!
If you followed the gentoo's handbook, you will have Gentoo's root partition mounted at /mnt/gentoo, so change to that directory (cd /mnt/gentoo) and type the following command to save the backup:
tar -cvjpf /stage4.tar.bz2 *
For example, if you want to save the stage4 tarball under /mnt/harddrive, the command will be:
tar -cvjpf /mnt/harddrive/stage4.tar.bz2 *
Now you can reboot your system, and start using Gentoo. For instructions to restore this backup when needed, see Restoring Backup section at the end of document.

Install Xorg and Desktop Environment
If your idea is to create a full backup of a complete Gentoo System, you must proceed with this method.

Just emerge everything you need and configure it inside this chroot (propietary display drivers and KDE, for example)


Compress the stage5
Once finished installing all Xorg and Desktop Environment related applications, it's time to compress the new and full stage5 tarball to have always a DE prepared Gentoo tarballed to be used in a future.
In order to do this, you have to mount the destination partition (must be other than where you installed Gentoo!) and run:
tar -cvjpf /stage5.tar.bz2 *
For example, if destination partition is mounted under /mnt/harddrive:
tar -cvjpf /mnt/harddrive/stage5.tar.bz2 *


Updating tarball
In order to keep stages up to date, you can update them in a running environment without the need to format nor modify your current installation.
In order to do this, you have to unpack your current stage tarball, chroot to it, update system, and repack. Step by step:

1. Create /mnt/stage
2. extract stage to /mnt/stage by executing: tar -xvjpf /stage5.tar.bz2 -C /mnt/stage
3. mount -t proc none /mnt/stage/proc
4. mount -o bind /dev /mnt/stage/dev
5. If you have any system folder in a separate partition, mount them (for example boot partition)
6. chroot /mnt/stage /bin/bash
7. env-update
8. source /etc/profile
9. After that, just run emerge --sync && emerge -uDN world
10. Once updated, we need to delete previous stage tarball and recreate. Now exit from chroot and do umount /mnt/stage/dev && umount /mnt/stage/proc
11. To repack stage tarball, go to Compress the stage* section

Restoring Gentoo from stored stage
In order to restore a saved stage when formatting is simple: just unpack it to where you want to install gentoo by doing tar -xvjpf origin/stage5.tar.bz2 -C destination and after chroot to it to update grub install (refer to gentoo's documentation for doing that)

I hope this guide will save a lot of time to anyone other than me. It isn't something new really, I didn't invent anything, only I wanted to post this for helping others too in restoring a gentoo's installation when something fails.

Wednesday, April 1, 2009

ext4 data loss

I've been trying out ext4 since a couple of months right now, specially since it was renamed from ext4dev to ext4, but nowadays I don't advise anyone to use it, at least until kernel 2.6.30 or so [1] under severe risk of massive data loss!!

The problem I had comes from when I changed several kernel option wich made my system completely freeze on shutdown without properly syncing disks, and I couldn't even reboot. fsck didn't help me either.

Read article [1] for more information, and take it into account before trying out. As a side note, it is "safer" to have ext4 on a NOT SYSTEM partition (safer does not mean safe), but not on such an active partition such system one.

[1]: http://www.h-online.com/open/Ext4-data-loss-explanations-and-workarounds--/news/112892

WoW Experience with SSD

I recently bought a Sony Vaio (VGN-AW21Z) with a 64GB Solid State Drive and it is just amazing.
For those who does not know what it is, it is a hard drive made with flash chips which improves read/write performance and battery durability.
What I noted more are tarball extractions, system load, compile times and it is definitely worth, but there is a huge problem: its price.
Even with that high price, I would recommend almost anyone to buy one, but NOT for massive storage use, only buy one 'small' to store the system and programs installed and other HDD for storage of data, that way one can save much money while enjoying the speedup :)

Wednesday, March 25, 2009

Gentoo Easy Handling

Firt of all a brief explanation of how gentoo works (a simplification of http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=3 look for a more extensive explanation).

Gentoo works with two kinds of software trees: the stable tree, and the unstable one; Gentoo's installer portage works with ebuilds which contains all installation data, and everything gentoo needs to install a package. Inside of it there are one thing of interest: the KEYWORD variable which can determine wheter that package is marked as stable or not.

Before starting to explain how gentoo works, it is interesting to mention how packages are handled, and how versions are handled first. Gentoo has a particular syntax for a package: "category/packagename", but for mentioning a particular version of a package, an operator is strictly needed, for example media-test/test-4.0.0 is incorrect, it should be something like =media-test/test-4.0.0 (you can use <,>,<=,>=,= or ~ operator, the ~ operator will select all revisions for a particular version, for example ~media-test/test-4.0.0 will select 4.0.0-r1, 4.0.0-r2, and so on)

On the other hand, we have the keywords values:

  • Stable, is the value (substitute arch with yours, like for example x86, amd64...) and is meant to be tested enough by gentoo developers and users which will not break things (hopefully). For installing stable software, nothing else is required, only to execute "emerge ".

  • Unstable means that the package is not tested enough and may harm your system (other packages depending of it don't work anymore, etc), and it is not advised to install. They are keyworded with <~ARCH> value (for example ~x86, ~amd64, etc). To install this kind of packages you have to include the line category/package ~arch to your /etc/portage/package.mask

  • But there are more, sometimes, there are certain situations a bit more special: when a package is listed in package.mask file, or also known as hard masked. This happens when a package is known to cause severe problems to rest of system or only if it's too new to be tested. To install this software, you must unmask it by adding the line "category/package" to your /etc/portage/package.unmask
  • There are cases where you need to manually hard mask a package to prevent it from beeing installed in an update (usefull when you mix between stable and unstable branches and don't want a conflictive package to be updated). In this case, it is enough with adding category/package to /etc/portage/package.mask
In order to simplify this task, I've created a script to simplify this task. This script is rather simple to be used, and it is autoinstallable, you just need to copy/paste to a file, and run (as root) ./install.sh arch (substitute arch your your architecture).

After installing, the usage is: gentoo [keyword/mask/unmask] and it will add the correspondent line to the correspondent file inside your /etc/portage directory.

I hope it could be usefull to someone else than me :) and maybe I could update it adding functions or correcting bugs.

Thanks

The Script:
--------------------------

#! /bin/bash
if [ -z $1 ]; then
echo "Use ./$0 [ARCH]!"
exit
fi

echo "Gentoo Easy Handling"
echo "By StormByte (StormByte@gmail.com)"
echo "Execute this as root!"

echo "
echo \"Gentoo Easy Handling\"
echo \"By StormByte (StormByte@gmail.com)\"
echo \"Remember to execute it as root or as user inside Gentoo group!\"

ARCH=$1

function usage {
echo \"Use gentoo [keyword/unmask/mask] category/package\"
exit
}

function display {
echo \"Package \$2 added.\"
}

if [ -z \$2 ]; then
usage
elif [ \$1 == \"keyword\" ]; then
echo \"\$2 ~\$ARCH\" >> /etc/portage/package.keywords
display
elif [ \$1 == \"unmask\" ]; then
echo \"\$2\" >> /etc/portage/package.unmask
display
elif [ \$1 == \"mask\" ]; then
echo \"\$2\" >> /etc/portage/package.mask
display
else
usage
fi;

" >> /usr/bin/gentoo
chown root:portage /usr/bin/gentoo
chmod 775 /usr/bin/gentoo
echo "Installed under /usr/bin, name of command: gentoo"
echo "For more instructions of how to use it, invoke gentoo without arguments"

--------------------------
End of the script

Sunday, August 10, 2008

Beware with MSN Tools

After receiving tons of spam from friends who argue not sending it, I decided to write something in order to help prevent missuse of MSN account information.

I've heard so much times from people arguing that someone has a virus that send links to some MSN related pages(third party ones), for example those pages which appears to tell you who is blocking you in MSN, but almost anyone knows that this is not a real virus, is even worse.

First of all, some time ago, Microsoft changed the way that information is accessed and it cannot be accessed nowadays with just your account like before.

Second, when you type your MSN address and your password, despite it is illegal, they can have since then access to your account, and what they do is connecting a bot to your MSN and send spam links to every of your buddies, so if you did this, or you are unsure, the best you can do is CHANGE YOUR MSN PASSWORD AND SECRET QUESTION __NOW__ and NEVER REVEAL YOUR PASSWORD TO ANY THIRD PERSON SITE OUTSIDE LIVE MSN OR HOTMAIL.

To sum up: it is better to prevent than to cure in those cases, as they could steal your account too by changing your password if they want to... so advise anyone NOT to use that kind of fake services.

Tuesday, August 5, 2008

KDE 4.1 SpeedUp and Workarrounds

KDE 4.1 is finally here and thus, everyone can go ahead and try it (I recommend doing so) because it is really improved and it does not have that huge amount of bugs and loss of implementations KDE 4.0 had.

If you red my last post about the KDE 4.1 MIRACLE, you'll notice I was speaking about a huge performance impact in processing some events. After much research and tests, I've discovered the cause of this huge performance impact I'll try o explain here in order to help anyone having those issues (specially Gentoo's users, since much of this is about compiling some parts with or without some special feature).

I've discovered two possible causes:
  1. For NVIDIA users, as is listed in so many sites, read and try: KDE 4 and NVIDIA problems.
  2. For everyone (specially Gentoo's users): This is really important! Check out that your distro's maintainers or you, compiled strigi with fam and/or inotify features enabled (in Gentoo, check for fam and inotify USE flag being enabled). In my case, just enabling fam and inotify for strigi made my KDE 4.1 trunk far more stable than it was. If it is your case, just add "app-misc/strigi fam inotify" to your /etc/portage/package.use file if you are using portage, or to your /etc/paludis/use.conf if you use paludis, and recompile at least strigi.
To finish, I only can say that now, I can say, and prove that KDE 4.1 is really a miracle for everyone's Desktop!

Friday, July 11, 2008

The KDE 4.1 Miracle

As of July 2008, we can have finally the first betas of what most of KDE users were waiting for: the betas of KDE 4.1 series.

I've been using KDE 4.0 series since it was in early stages, and I've been keeping track of all SVN changes by rebuilding it at least two times per week, and trying to report as much bugs as I could. I must admit that it was (and *is*) a really exciting thing to have svn installed, as they burn it with changes at least every two days. But the wait isn't over yet, there is MUCH TO DO to improve speed of KDE 4.1,as it became already incredibly stable in these days. But... What about speed? The bad news about KDE 4 world nowadays is speed issues, it is true that a few seconds (two or three) passes since you click and the action is taken, and I could not tell what is the cause (KDE4 codebase, QT4 code, so much bufferings of transparency of plasmoids....) but I really know they CAN make it although maybe not for 4.1.x.

But leaving apart speed issues, I think KDE4 is definitely revolutionary, as you can see in several images all over the internet (including these two I published). Everything has evolved relatively quickly: almost every important program is ported to QT4/KDE4, or is being ported. The result is a whole new Desktop experience one can feel from login to logout.
To mention a few of KDE 4.1 features in the first image: Amarok2, with that new designed interface and look while maintaining its power; the "Folder View" plasmoid, which is what most of we were looking for and missed in 4.0; the redesigned panel with its plasmoids (clock, taskbar...); the transparencies in Desktop's plasmoids...
But not all in KDE 4.1 are plasmoids, it comes with a huge suite of programs (as it always had, some of them are redesigned, and some other are new) ported to new codebase, which makes it more usable. To comment only a few of them which can be seen in image 2: The Dragon Player, an excellent written from scratch video player; Dolphin, the new file manager, which counts with more features that konqueror had, like split view; the new Krunner interface (triggered by default with ALT+F2) to execute commands, and even more.

On the other side, just to mention why KDE 4.1 will be good for integrating programs are SOLID and PHONON which still need to be improved, but could show nowadays the power of the near future of KDE: While solid abstract programmers from all hardware access they could need (without bothering programmers to write code for hardware access), phonon does the same with audio related issues, and thus, allow programmers to spend their time in what is really important: their programs.

To sum up, I am just adventurer to install from SVN, but I invite anyone to go and try the beta, and of course, to install 4.1.0 when it's ready (near July 22th).