Keigo, or the Japanese art of "polite conversation", was my worst nightmare throughout my first two years in Japan. Using the appropriate Japanese in the appropriate situation takes weeks of studying, months of practice and several embarrassing moments to get it right. Fortunately for yours truly, any bloopers I might have made were promptly laughed off as something "typically foreigner"; for your average Japanese person, however, using the wrong kind of language can amount to a capital mistake.
The Japanese language has several levels of "politeness", ranging from casual Japanese (where one is allowed to use the verbs in plain or dictionary form) to polite Japanese (using formal verbs ending in -masu) and finally keigo, which is the ultimate linguistic tool in lowering yourself and praising your conversation partner(s) at the same time. To make things a little easier to understand, if I were a company employee, I'd use casual Japanese to speak to my wife and kids, polite Japanese to speak to co-workers on the same level and my immediate boss, and keigo to address the head of the company.
Let's take a closer look at keigo. Keigo can be divided roughly into sonkeigo (尊敬語, respectful language) and kenjogo (謙遜語, humble language). When talking to a customer, a superior or an elderly person, one must use kenjogo to lower oneself and sonkeigo to butter up the other person. Phrases* such as "I am very thankful that you were benevolent enough to--", or "It is a frightful thing to humbly inform you that--" are fairly common when using sonkeigo. Kenjogo, on the other hand, has fairly softer connotations in comparison -- one must simply use words that hold the same meaning, but supposedly show less consideration for oneself.
Another peculiarity of keigo is that of "honorific" prefixes ("o-" and "go-"), and suffixes ("-sama"). These can be added to virtually anything... which can lead to fairly amusing situations where a bank clerk might ask for "your honorable address and phone number" (御住所, go-jusho and 御電話番号, o-denwa bango) or a pet shop salesman might congratulate you on "your honorable and most respected dog" (御犬様, o-inu-sama). The fact of the matter here is that anything that has to do with whoever has a 'higher' position in a conversation is to be addressed with the utmost respect. Quirky, isn't it?
Let's dig a little deeper.
As far as I know, humbling oneself when talking to someone of a higher social status (whether it's a superior, a teacher or a customer) is a centuries-old tradition in Japan. Back in the day, if you dared to use the wrong kind of speech, a samurai was free to chop off your head and be on his merry way. Nowadays, you won't get beheaded, of course; on the other hand, you might earn yourself a social stigma, and it will take years of hard work and humble service before people might finally stop seeing you as "that guy".
In this day and age, keigo might seem like an unnecessary hassle. Keep in mind, however, that Japanese are a people who care dearly about appearances (sometimes, a little too much if you ask me). The purpose of keigo is to show respect. Whether you actually feel it or not is another story. As long as you use the right forms at the right time, neither you nor your conversation partner run the risk to "lose face".
In other words, your employee might curse your entire family tree every night before they go to sleep; chances are, you'll never know. Unless, of course, he tries to kill you, in which case you (should you survive the attempt), your company and your local community will probably be extremely shocked at the news.
*These are, of course, approximate translations, but you get the gist of it, right?