andrelop's blog

I’m widely know (by my peers, not globaly know, as you may guess) for being a well controlated person when it comes to spending my sacred and beloved money. Actually, I’m so well controlated that I’m sometimes seen as someone who hates to spend money.

Sometimes I’m almost convinced that people thinking that way about me are right, but then I get back to my conscious mind and realize that they think like this because people tend not to agree with what they don’t practice and, generally speaking, society has been continuosly teaching them to become as consumists as possible.

Obviously, I’m not completely protected against consumism, as I’m human and have as much desires as anyone else, but I think I’m doing well on keeping myself from being taken to the path which lead us to bit the bullet and spend our money on something.

What contributes to this control also demonstrates how we’re used to learn better only from mistakes, as I already spent too much money on things which I initially thought I desperately needed, but which after I realized weren’t actually all that much needed after all.

I have been trying to avoid wasting money on an e-book reader for months. At the beggining it was a piece of cake, as there wasn’t a real option (mind you that I live in Brazil, not in the USA or some other so called first world country). However, in the last few months, the scenario has been changing and it seems that these days there are actually some options available.

I have been researching privately about options for months and have become well aware of all the advantages and disavantages of all the options available today. Sorry, I won’t be pointing the right option to you, as your best option probably will be different from mine and recommending things isn’t the point of this post.

Technically speaking, the options available today seems to provide me with what I need. There are no doubts that almost any of them would be good (but not all of them would be “the right”) choices and actually would represent a real improvement over the current situation.

So, one might ask, why not go ahead and just buy the damn thing ? For most of the people out there, it would seem to be the right thing to do, as I would be doing conscious and well researched purchase after all. Well, I would be doing something good for myself for sure, but the government actually would need to show me some respect and prove me they have some respect for me as well.

Amazon, for example, is shipping the Kindle for other countries and Brazil is one of them. I would surely just go ahead and buy it the day the shipment was announced and that was almost what I did. What prevented me from doing so was the insane/absurd takes government is applying on anything one wants to import from another country.

Even after converting the price from dollars to the local currency (reais), the price is something like three to four times higher that the original price in the origin country. It’s insanely prohibitive for the vast majority of citizens and one could wonder why this is so when this is a device used basically for reading. Mind you that the government has big tax reductions for importing books, for example, which are seen as culture related items and so get to win some advantages over other non-culture related items.

To summarize it all, if you want to convince yourself how not to spend money on something, even when you really want to, just do the math regarding the amount of the taxes you would need to pay and I assure you there will be no regrets when latter you think why you did not bought it.

Recently I’ve been dealing with requests for LDAP directories configs which clearly demonstrates how little people understand about directories and what they are capable of doing. A significant amount of people I know seems to really get it wrong when it comes to what they want from a directory.

Actually, they somehow think that a LDAP directory will do whatever is needed for their application to store, retrieve, validate, authenticate and even take decisions based on no data provided at all. These people think that one should just deploy a directory using the minimum effort approach and suddenly everything will just work.

They don’t seem to realize that for their application to make use of a directory it should be prepared to do so. They can’t accept it when they are told that the directory won’t just work and by some unknow enchantment get their systems data stored, validated, authenticated and, shockingly for them, that it won’t make their credentials consolidated so lots of different services will out of the box just start working using the same username/password pair.

Also, some people don’t understand the difference between a LDAP directory and a single sign-on (SSO) system. They don’t realize that a directory won’t, by itself (i.e. without additional software and some respectable amount of tweaking), provide them the ability to authenticate against it only a single time and have their credentials shared among all their systems.

That’s it. Said. Don’t get me wrong. All that was metioned above is possible, but it isn’t done by LDAP alone. LDAP is just a bunch of protocols and a directory is only one nice place to store information. What will be done with this information, how it wll be treated and how it could be used to produce meaningful results are almost always up to the application and/or to some “middleware” or added plugin/overlay/connector/whatever.

Next time someone ask you to “install LDAP so I can get rid of all my different username/passwords and use only one instead”, be afraid. Be very afraid and present him/her some theorical knowledgment regarding the topic. Or, better said, insert some clue into his/her brain.

I wanted to let my readers know that I migrated from my previous hosting to a new one. There was nothing wrong with the previous one, but I wanted to go out shopping for something cheaper and still as reliable.

Lots of friends gave me good recommendations for Linode and the price was tempting so I thought : “Why not ?”.  The most noticeable advantage, apart from the price reduction, was the speed improvement, as I got myself a more powerful VPS and then now I have more room for new experiments.

Later I also noticed some little but very welcome improvements. These are just minor details, but together they account for a great experience. So, I’m here to let you guys now that if you want a good, reliable, speedy, and still cheap hosting provider for your VPS, just give Linode a try.

I’m sure you won’t be disappointed.

Since the begginings of times, people have been trying to come up with new ways to become smarter, leaner, cleaner and sexier. Surely, not having to do anything to actually get to this nirvana state would also be a more than desired plus.

Personally, I also tend to want to get some of these niceties myself, but at least I know that it’s not that easy to come to a master state, specially because I’m not at this Jedi state on none of these areas.

Back when I was starting to learn my way through all this tech stuff, I realised that if I really wanted to master something, I should do my homework and try to find out a way to learn by myself, not having to rely on someone else for everything.

Sure, there are those really smart people, which we often call our “gurus”, and to which we could try to resort to when things aren’t really working the way we had been trying to make them work.

The sad reality is, by being “gurus”, those really important and highly requested beings, these people are really illuminated and have no time to answer to each and every silly question we would like to ask them so we must find our way somehow and only resort to them as a really last resort, when everything else fails and there’s no hope anymore.

That was fine with me as I always liked the feeling of discovering new thigs by myself, understanding how things really worked and how they could be driven and manipulated in order to accomplish goals which would please me. Besides, this is fun.

By doing this, I learnt a lot and still am learning more each and every day. That’s “The Right Way”, as they say, and really is how things should be … except if you are a newcomer these days, it seems.

I’m by no means a “guru” and would surely not cassify me as one, but I think I have talked to many of these mythological beings over the years so I can tell you what you shouldn’t be doing when trying to approach them.

First of all, please, pretty please, try to do your homework before resorting to more knowledgeable people about a given subject. The more knowledgeable people would surely be pleased to help you if you show them you deserve the right to be helped.

When starting to work with a technology you do not master or even don’t know a thing about yet, the knowledgment about such a tecnology wouldn’t magically be transfered from your local guru’s brain to yours.

Even if your local “guru” is a nice person and get to waste some sentences with you about such a tecnology, do not pretend you can master all about something only by listening to some words from him/her for a couple of minutes.

Well, let’s say that if your guru can give you some initial hints without you firstly demonstrating him/her you did your homework, take it as just it : hints. Follow these hints as clues so you can start researching about the given subject and then later show him you was able to learn a lot by yourself.

Your guru will be pleased and surely will start considering you as a good padawan, a padawan worth the time spent explaining things to. The universe will start doing its magic and things will start working for you, who will also start feeling good for being admired by your peers.

Yes, that’s how “Show me the way I should follow” works. You know, “Show me the way I should follow” isn’t “Do all my work for me”. The later happens to be called “consultancy” and can be arranged for a negotiable amount of money between the involved parts.

Also, please, don’t take it as personal when someone doesn’t want to give you a “hint” about something. Remember that he/she could be really busy, not in the mood or simply exercising his/her right to ignore you if you don’t show him/her you are worth the time he/she will spent explaining things to you.

Personally, I refuse to reply to people who consistently try to use me as some sort of human search engine. Even if I do know the answer for some question, most of the time I try to Google for it before replying to someone who asked me about a given subject.

If I find out that the answer for a question someone asked can be found easily between the first and third hit returned by Google, I happen to ask the person who asked me the question if he/she really tried to research a bit about the subject before asking me about it.

The answer is a strange mix. Some people say the truth and tell me they asked me because they are in a hurry and couldn’t afford the time it would take them to search for an answer when I was readily available quickly. Sad, but at least they are saying the truth.

Some people try to lie and tell me that they tried to find an answer and had no luck despite researching for a long time. Strangely, they seem to stop the conversation right after when I show them that putting the term on Google’s search box and hitting the search button would bring the answer for their problem as the first hit.

The truth is that if you get all the answers for free from your local guru, you will never learn how to look for answers by yourself and will indefinitely depend on someone else to get your job done. Not a pleasant situation, even if you are silly enough expecting to live like a information sucker for your entire life.

When you get in a situation in which you have no workmates or available friends at hand to help you, you won’t be able to accomplish your tasks. Your boss then won’t be nice when that happens and he/she will surely start considering replacing you with someone else.

The opinion of your workmates, which by now could easily see you as a major source of loads of lost time, won’t help you keep your job either. You can be lucky and get the confidence of your workmates, but be sure that management will notice and you won’t last too much on your current position. Gurus have the tendency to be good at finding these kind of people and management also have the tendency of listening to gurus. Do the math.

Do the right think from the beggining. Do your homework, work out your own way, try as hard as possible to find out solutions for problems by yourself and only resort to your local guru when there’s no way for you to go forward about solving a given problem by yourself.

Also, even when these situations come up, try to show to your local gurus that you did your homework and explain clearly to them what you have managed to find out and where exactly you are having a hard time progressing.

This will surely help not only your local guru to help you best, but also surely will get him/her to take you as a really smart person, a person which is worth losing time with and worth  adopting as a future padawan.

It’s no secret for those who follow me on Twitter/identi.ca that I have been toying with virtualization technologies for some time now.

I have been using Xen in production for some time and, apart from not thinking it’s the way to go for future virtualization, it’s quite usable right now, specially if you are going to use Debian Lenny as the dom0, as it now has a more modern kernel than the dreaded 2.6.18 we Xen users were forced to keep using.

However, I have been keeping my eyes on KVM as well, as I always liked how much simpler and well integrated into the Linux kernel it is. I’m even subscribed to the KVM development mailing list, even not understanding most of the things the developers are talking about there. It’s no problem for me, I just want to know what’s being worked on and what’s on the pipeline.

As a KVM fan, I’m also using it intensively as a tool for prototyping servers. It’s easy enough to set up a new Debian server to test things on and learn new technologies, as well as troubleshoot problems without intefering with production environments.

Althought KVM is not as good as Xen when it comes to performance, it’s quickly improving every single day. And, for those of you who still believes KVM is only about full virtualization, I’m happy to say that it has come a long way and paravirtualization has started to infiltrate KVM land as well.

Today, KVM already sports paravirtualized clock (pvclock), paravirtualized memory management unit (pv MMU) and VirtIO drivers. Actually, it seems that KVM these days is the biggest user of VirtIO, maybe only loosing the leadership to Rusty’s lguest.

By using VirtIO’s disk and network drivers (virtio_blk and virtio_net, respectively, and its associated modules), KVM can deliver a much improved I/O experience than when using QEMU’s emulated drivers. For this reason, it’s always preferable to use VirtIO drivers whenever possible when setting up your KVM guests.

During the Etch lifetime, one will need to do some tricks in order to use VirtIO when using Etch as a guest under KVM. However, when preparing Lenny’s d-i, the developers were smart enough to add to it virtual disk detection support. What it means is that now, starting with Lenny, d-i will recognize that it’s being given a VirtIO block device and automatically load the needed kernel modules to support it.

Also, the disk detection and partition modules (partman et all) were modified to show a detected virtual disk (i.e. /dev/vdX) and let the user partition it, as well as grub-installer was changed to allow GRUB to be installed onto a virtual disk’s MBR,  effectively making d-i a really powerful virtualization aware installer.

Here you can see Lenny’s d-i showing a detected virtual disk, named “vda” :

Brazilian Portuguese readers could see the version using brazilian portuguese texts on the screen here. Sure, d-i also received a lot of improvements and special support for installing Lenny as a Xen domU was added as well, but I haven’t played with it yet, so I won’t comment on that right now.

And, hey, one can even use virt-install and virt-manager to deploy KVM guests under Lenny And you know what gives me even more confidence that KVM will be a first class citizen inside Debian ? The fact that Steve Kemp is toying with the idea to change his xen-hosting.org project so it would become a new kvm-hosting.org project.

Maybe I’m praying for the preacher here, but Steve is very well well know for being the author of xen-tools and xen-shell, as well as being the creator of a number of other nice free softwares. And, judging by the comments on his post about the future kvm-hosting.org project, it seems that Steve maybe will need to update xen-tools and xen-shell to account for KVM or create a new set of tools dedicated exclusively to KVM based on his past experience creating the current tools for Xen.

Good times ahead for those of us who are surfing the Debian Virtualization wave, indeed

Here I’ll try to present you some tips on how one could easily make a playground become a paid fun. And no, I won’t be teaching you how to become rich, although I would like if you would teach me how that could be accomplished as I’m trying to come up with a way to do just that for a couple of years now and failed miserably.

Recently, I’ve been setting up KVM virtual machines in order to do a lot of testing for a project I’m involved with. Letting boring details aside, one could say that a very long, boring and error prone setup was finished after a lot of work and it worked like a charm in the end, as a KVM machine.

As always, laziness is a given and when you are into it you empower yourself to come up with some nice hacks and goes that extra mile to find out how one could not to have to do boring and repetitive things again and again, in the best spirit of “let’s have some work now and save me ten times more work later”.

As I’m becoming more and more into that spirit of laziness, there I went looking for a way which would let me transform my beloved and nice working KVM machine into a fully featured real physical server. After some tips from microblogosphere friends (thanks fike), I had some ideas on how that could be accomplished.

Turned out my ideas weren’t really all that right so there I went again looking for a way to do what I wanted. After some research I found out that the KVM guys already gave some thought about it and even had a ready solution to my problem : it’s called qemu-nbd.

Yes, by now everyone should know that KVM builds on top of the excelent (just not so snappy) QEMU. However, as I’m a Debian user, here I should say that in Debian, most notably Debian sid/unstable, the qemu-nbd binary was renamed to kvm-nbd, just as the KVM binary is called only kvm and not qemu-system-x86_64 or whatever else is called these days upstream.

NBD (short for Network Block Device) is a nice thing. I hadn’t tried it before (I certainly used and am still using in some projects of mine the not-upstreamed-yet-but-so-nice-that-I-couldnt-resist DRBD, which seems to share some ideas which NBD). In short (read NBD link for more info on that), NBD allows the Linux kernel to use a remote server as a block device. Also, its upstream is a fellow Debian developer (hello Wouter). How nice’s it ?

As NBD is an upstreamed kernel feature, you don’t need to go to the external-module-route-hell. You only need the userland tools, which are nicely packaged and integrated into Debian as well. They’re only one aptitude away, so you could use this command to bring it to your KVM host :

aptitude install nbd-client

After that, you will want to export your image file (the file which represents the disk of you virtual machine) as a NBD block device, so you will be able to mount it under your KVM host and do whatever you want to do with it. Here’s how one would do it :

kmv-nbd –conect=/dev/nbd0 myimagefile.img

I tested it using both qcow2 and raw disk images and both worked like a charm so I don’t think you will have any problems here. Next, you need to create a directory structure under your KVM host which temporarily will be used to mount the partitions inside the virtual block device of your KVM machine. You could do this like this :

mkdir /newserver

mkdir /newserver/usr

mkdir /newserver/var

mkdir /newserver/home

In the example above, I’ve created these directories because I want my new server to have separated root, /usr, /var and /home partitions and mount points. Feel free to adapt to any kind of partitioning layout you please.

Next, you will mount yout virtual machine filesystem under a temporary location, so you will be able to copy all of its content later to your final desired destination, i.e. the real partitions inside the real disk you are going to use in your future real server (not a virtual machine anymore).

I did it using :

mkdir -p /mnt/temp

mount /dev/nbd0p1 /mnt/temp

Notice that the /dev/nb0p1 above is my root partition inside my KVM machine, only that it’s represented now in my KVM host in NBD’s device node notation here. I’ve only one partition inside this particular virtual machine (aside from the swap partition).

Next, you want to mount the partitions on the real disk you want to be used in your new real server under the directories created earlier so you will be able to chroot to then later and do any tweaks you may want to, like fixing /etc/fstab, /boot/grub/device.map, /boot/grub/menu.lst and all these places we know makes references to block devices which may differ from the situation you had in your virtual machine running under the KVM.

Here’s what I used :

mount /dev/sdb1 /newserver

mount /dev/sdb3 /newserver/usr

mount /dev/sdb5 /newserver/var

mount /dev/sdb6 /newserver/home

The device nodes above are from your real disk’s partitions, not from any virtualized device block access method like NBD. Also, take care not to mount your primary disk’s partitions (the ones you are using under in your KVM host) as you could destroy them easily. As you can see above, I’m using /dev/sdbX as /dev/sda is my primary disk, the one under which my KVM host is running. Again, adapt to your scenario.

Next, you just copy everything from your virtual machine filesystem (which is now accessible from within your NBD block device) to your real disk. I did it using :

cp -a /mnt/temp/* /newserver/

After that, you can disconnect your KVM virtual machine image file from the exported NDB device block using :

kvm-nbd -d /dev/ndb0

And then chroot to your real disk layout yout just copied to /newserver in order to fix anything which would make reference to block devices while running as a virtual machine under KVM but which now will be a completely different thing as a real server, like your /etc/fstab, /boot/grub/device.map, /boot/grub/menu.lst and so on.

In order to chroot to your new soon-to-be-real filesystem you could use :

chroot /newserver

Then go nuts and start fixing everything you may think needs to be fixed so this filesystem could be used in a real server to boot it up. Im my case, I needed to fix /etc/fstab in order to mount my additional partitions (as under KVM I was using only a root and a swap partition and now, under the real server, I’ll be using separated root, swap, /usr, /var and /home).

Before I went changing more things, I “fixed” my /boot/grub/device.map and my /boot/grub/menu.lst files to point to /dev/sdb and /dev/sdb[1],  respectively, so I could run grub-install from inside the chroot in order to install GRUB into the real new disk’s MBR (master boot record), using :

grub-install /dev/sdb

Then, finally, I “fixed” /boot/grub/device.map and /boot/grub/menu.lst again so they would point to /dev/sda and /dev/sda[1], as /dev/sda will be the device node for this new disk when it wil be running under the new real server and not as a disk inside the KVM host machine.

Next, I exited the chroot using :

exit

And umounted the new real disk’s partitions, using (now already out of the chroot) :

umount /newserver/home

umount /newserver/var

umount /newserver/usr

umount /newserver

Now you can take this new disk out of your KVM host machine, put it inside your new real server, physically install it there, turn on your new real server and everything installed while the filesystem in it was being used under the KVM virtual machine will be there, running nicely.

And you’re done. That’s it. Also, it’s important to point out that it’s much easier being done than being said (or written, in this case) so, if it looks scary because of the size of this post, don’t let it disincourage you as this whole thing is a pretty straightforward procedure, much easier than it seems.

And as I know that I will receive lost of complaints from people which will tell me how inefficient this whole thing is and how I could have accomplished the exact same thing in a much easier and faster way (perhaps even with a already existing tool which would automate almost entirely the whole proccess), I would say to these people to please exercise their right to share ideas using the comments. Just be nice on me, please.

After all, it was fun and I learned new things while doing it all, as well as I tried to share what I learned from this experience with my peers. That’s what matters. Peace, love and geekness to all of you

aba, if you happen to receive some feedback on this subject, please let me know. A comment in this post or even a blog post from you in Planet Debian would be great.

Bloggind as there’s no comment support available in your blog.

Sorry Planet Debian. The last post was intended to appear on Planeta Debian Brasil only, not on Planet Debian. I tend to separate my posts written in english and in portuguese using different categories for each and have both Planet Debian and Planet Debian Brasil being feed from the relevant URL for each category.

It was just a mistake from my part giving the last post (written in portuguese) the wrong category. It’s already fixed and I hope not to make the same mistake again in the future.

Yes, it’s true. Advogato is going offline. It’s sad to see such a good project going offline. I started using Advogato and reading some diaries there a lot of years ago and also wrote some entries myself there.

Anyway, thanks Advogato for all the fish and good luck.

Nice to know that my friend Otávio Salvador started his English blog. As he noted, his toughts will be sindicated on Planet Debian soon. It’s always a pleasure to know which are the newer crazy ideas he’s working on. Go Otávio !

Update : I’m also blogging in Portuguese. My friends from Debian which can read Brazilian Portuguese can read my ramblings on my blog. With some luck, my ideas will soon also be available on Planeta Debian Brasil, a Brazilian planet for Debian lovers.