Remember last week me talking about how the bandwidth provider for my company decided to be just plain stupid and put the screws to my company, thus ensuring we would hate them forever? Guess what? They weren’t finished.
This morning, as I was walking out the door, literally, with one foot on my patio, I get a call from my boss. It went something like this:
He: “Did the domain transfer screw up our network name resolution?”
Me: “Not that I can think of. Let me check the transfer status.”
He: “Because we have no communication, again. No email, no web, no vpn.”
Me: “It looks like the domain name is still in the previous registrar approval phase. I am not sure what is causing our outage.”
He: “Could you call our sys admin to see if you can figure out what is happening. Right now we have no email, no web, no vpn.”
Me: “Sure thing, let me call him.”
I called my guy and together we came upon something a little different from the last outage last week. We started with our normal troubleshooting techniques: ping, tracert, dig. Very quickly we came to the conclusion that our DNS was not resolving again for some reason. The only difference between this time and the last time was that this time the DNS servers were not responding at all to our requests. I called my boss back and told him that I had given our sys admin the name and number of someone to talk to, even though my attempt at calling him was an epic fail. The phone went to voice mail, but they have not message center so I could not leave a message. Instead I was transferred to the switchboard where I was told that there is no known number and I was hung up on. Nice.
Knowing that my hands were tied at home I told my boss I would be in shortly and began my drive to work. When I got in about 30 minutes later my boss was on the phone. Wanna know with who? You guesses it. Our bandwidth provider.
Now I know you want to know this. What was the cause of this outage? Was it a catastrophic failure of a hard drive? Perhaps a full blown upstream network outage? Maybe a power outage? No. You’re going to love this…
We didn’t pay our bill.
What? Wait, we just took care of that last week didn’t we? Apparently not. See, the email that contained the invoice we were supposed to pay from never sent. Their paper invoice printing system has been offline for a few months so they could not send us a paper invoice. And apparently it was not noted anywhere in our account that we had spoken to their highest level AP person who had assured us that our services would not be terminated again. With that, someone who saw that our account was past due decided to pull our plug all over again.
Are there any further lessons to learn from this? Sure. Try not to be stupid more than once in a given weeks period of time. I tend to think that if that company were a person my boss would have driven to that person’s house and kicked that person’s butt halfway across their bandwidth pipe. Which might not be as far as I think seeing as they are now pretty much nothing more than a name and a phone number. That doesn’t allow messages or actually route to anyone.
I really don’t like starting a Saturday working on work stuff at home. Especially when I am not supposed to be working on a Saturday. I don’t mind working when I am supposed to be working, but when I am supposed to be making breakfast for my family or taking care of stuff at home I get a little testy that I have to interrupt that so that I can tend to things at work.
So it went this morning, when by 8:45 I was getting called on my home phone and cell phone that there were things breaking left and right and why was this happening and what is going on? Huh?
It started with a frequent background job failing but not notifying. It then led to a series of other issues including an intranet that went down, an account management app that went down and our core site that went down. Coupled with that were all of the background jobs and middleware apps that were not transferring data as expected. Things were not working so me and a few fellow co-workers were.
The cause of the situation ended up being a few other nightly jobs running, killing something and never bringing those somethings back up. Then another job decided to reboot a server but not bring it back up properly. This in effect made all of the other servers that talk to the one that was reboot to stop talking to it. It happens. Anyone that has ever worked in an IT environment knows things like this happen.
But when handled properly it only happens the same way once. A second time means you didn’t learn from the first. Screwing up is inevitable and should be handled with grace and understanding the first time it happens. Any subsequent (and by that I mean the second time only… after that more than one person is to blame for these issues) instances of failure like this indicates an inability to learn from your own mistakes and should not, in my opinion, be handled with the same amount of grace and understanding as the first time. I know that may sound harsh, but that is the reality of living in a real-time environment in which money transacts and businesses serve.
The incident this morning was a case in point. Nothing that took place this morning should have taken place this morning. In fact, a remedy for this situation was put in place, for a different identical server, just a few days ago. So it begs the question, if an identical machine bought at the same time experienced something like this and was fixed, why would it’s twin not be fixed? You get my point.
What ended up happening is that three people, one of whom was me, ended up spending non-work time working because of an error on the part of another teammate of ours didn’t learn from the last time something like this happened. Yes, I am complaining about a teammate. We all are, or at least should be, held to the same level of accountability in everything we do as a unit. And this does not just hold true for work. It can easily be applied to sports, families, friendships and businesses.
I love my job. If I come across in a way that indicates anything otherwise please don’t hesitate to call me out on that. I actually look forward to coming into the office and taking care of business everyday with the team I work with. I appreciate their knowledge, their expertise and their experience. This is very much like my family, and more specifically my wife, whose experience and knowledge provides a wealth of protection to me and my family. My team at work provides a similar level of protection to our team at work.
That is the nature of a team environment. We protect us. We advance us. We are us. Every person on the team should take the responsibility for the success of the team and do whatever it takes to maintain a high level of accountability to the team to see to it that the team succeeds. If that is done then Saturdays can be spent as a normal Saturday and my blog post for today could have been about something entirely more pleasant than this.
Today I had the misfortune of having to migrate a website from one server to another for a sister company of ours. I say misfortune because this website is coded in the nastiest bit of Cold Fusion code I have ever seen. I mean it looked like a script kiddie that just learned how to regurgitate CFML from the back of a cereal box tutorial wrote it. And I had to make it work on a new server.
Did I mention that I haven’t been in Cold Fusion for over five years? And I hated it then.
Good thing that I am a fairly adept PHP developer. Instead of futzing with crappy CFML I decided to spend a few hours trying to port it to PHP in a sensible way with sensible coding and sensible architecture. The result was that I built a mini-MVC framework for it in about 4 hours. Aren’t I awesome? Well, isn’t PHP awesome?
I can do a lot of things in PHP typically in a short amount of time. I can write test snippets, mini-apps, middleware, all sorts of goodies fairly quickly when I need to with this language. And what I write generally is easily understood (comments anyone?) and easily maintained. I cannot say that about that pile of crap that was the Cold Fusion site. That stuff stunk like butts on a hog. But I shall try to not focus on that so much as tell you how much I love PHP today because it let me create a nifty little mock MVC style framework in about half a day.
Well, actually it is more of a VC framework with a small data handler object that talks to MySQL only. But it gets done what is necessary in an OOP style and is very easy to understand and maintain. Which is what I want.
Now all I have to do is port their templates and CSS over to something cleaner and I will be able to get their site off that vomitous mass of Cold Fusion Crap and on to something a lot prettier, easier to manage and, in my opinion, just plain better.
PS And if you are ever going to make a static HTML page for a single row of a database table, do not, anywhere in your code, set the row ID of the page information as it comes from the database as a hard coded numeric valued variable, then check if that variable is numeric, then pass that to the database as a query to get the page information of the page you are on. That is just stupid, plain and simple. Be smart, develop smart and use your brains. That is why we have them, to keep from making stupid mistakes that other, smarter, people might laugh at you for.
This past Monday I had the fortune of attending my very first meetup. I had set a goal for myself a few weeks ago that I would attend one before the end of the year. It happened way sooner that I thought it would and resulted in me being able to mix it up with a few fine folks at the Silicon Valley MySQL Meetup.
There was a business need for me hitting this thing up. Firstly, I am a web developer. I am not a DBA and I am certainly not a MySQL guru. I can write queries but the extent of my MySQL knowledge ends right about there. Secondly, I am responsible for management and maintenance of all of our MySQL servers at work. There is one chief DBA who is a Sybase queen, another two or three folks that know their way around a Sybase server (and to a lesser extent a general database server) and then there is me and my colleague who write web apps and MySQL queries. Thirdly, I am an administrator for a very popular PHP developers forum and knowing how to get myself out of the stupid ass scraped I have gotten us into when it comes to MySQL would be darn handy.
So I set out to hit this meetup. It was held at the Sun Microsystems complex in Palo Alto (or Redwood City or wherever you are when you come off the Dumbarton Bridge on the Peninsula side). I was looking forward to finally meeting a man that I have been communicating with for some time now (yes, you Don) and to networking with other MySQL/PHP/Web developers in my local geographic region. And I was not disappointed.
The talk itself was a little boring to be honest. But that is because I am not at all interested in database shards and the whatnot. However I was very interested in meeting Don Ravey, a fellow moderator on our forums, Mr. Tish Wood, a very prominent member of the PHP Meetup community and one of the coordinators of the meetup Eric Bergen from Proven Scaling. Not only that, but there were a host of other people there that provided excellent commentary, questions and challenges for the speaker of the evening.
I love being in the mixed company of brilliant people like that. It is humbling, exciting and gives me something to look forward to. I so enjoyed being there, connecting, talking with people, meeting a few folks and hopefully being able to glean something from someone somewhere. I just hope that I can, at some point, make it to another meetup. Perhaps a LAMP meetup or a PHP meetup. After this last meetup Tish asked me if I would at all be interested in speaking to a group at the Greater SF Bay Area LAMP meetup. I think I would love to do that.
If I can find an evening to get away for a while longer than this last evening. And if we can find a way to not have a meetup in San Francisco. Because as much as I like socializing and hanging with other technology professionals, I cannot fathom the thought of heading to The City for a 7:00 PM meeting on a weekday evening. Until then though, I can start thinking about what I would talk about if I were to ever talk.
And hopefully I will be a little more relevant to a n00B than that fella at the MySQL meetup was to me. He was good and he knew his stuff for the most part. But it was not my cup of tea so I sort of lost interest a little in the subject matter. That was ok though, because I met people, got numbers and had a free coke. In the end, what could be better than that?
I realized in my quest to build a new theme for my blog that as I did validation testing on the markup that it didn’t validate (against the XHTML 1 Strict DTD - because I am a little Geek OCD when it comes to my markup). After a bit of inspection I realized that it was the stupid YouTube code that I copied from their site that was throwing off my validation.
Apparently YouTube thinks that everyone that puts their code into a site writes their markup under the transitional DTD. How stupid is that? A la:
<object width="425" height="344">
<param name="movie" value="http://www.youtube.com/v/Rw8gE3lnpLQ&hl=en"></param>
<embed src="http://www.youtube.com/v/Rw8gE3lnpLQ&hl=en" type="application/x-shockwave-flash" width="425" height="344"></embed>
</object>
Well I wasn’t about to be told that I cannot have valid XHTML in my code. After all, I took the time to make sure the core code was valid, why should I tolerate invalid markup injected into my code by YouTube or Wordpress or anyone? Exactly.
My solution? Simple. Google.
And Google, being the friend to me that is always has been, helped me right from the go. It took me Valid XHTML to Embed a YouTube Video. After reading some of the goodness on this tutorial I read the comments. And lo and behold, I ran across another tidbit of goodness that all bloggers that are into valid XHTML output should look at.
The Valid XHTML YouTube embed code generator - Online tool is totally awesome. It is fast, clean and reliable. I have already used it twice and will more than likely be using for every video embed I post until I want (bad enough) to write a Wordpress plugin. Until then, if you are at all interested in keeping your XHTML template standards compliant, hit this little converter up because it rocks.
And just so I can prove myself to not be a liar… remember that little YouTube generated code snippet above? This is how it comes out afterward:
<object type="application/x-shockwave-flash" style="width:425px; height:344px;" data="http://www.youtube.com/v/Rw8gE3lnpLQ">
<param name="movie" value="http://www.youtube.com/v/Rw8gE3lnpLQ" />
</object>
<div style="font-size: 0.8em">
<a href="http://www.tools4noobs.com/online_tools/youtube_xhtml/">Get your own valid XHTML YouTube embed code</a>
</div>
Yeah, I thought it was great too. Enjoy.
A recent Sitepoint article by Rachel Andrews, Director of edgeofmyseat.com, outlined some pretty Nifty Navigation Tricks Using CSS. Anyone that wants to learn to make some pretty cool tabbed, button or vertical bar navigation lists should give this article a read through. It is a pretty well written article and has a great deal of code that can be easily copied and pasted for your development pleasure.
For four pages it reads very fast. It is easy to follow and the examples are practical. I wish there were some working samples of the code, but still, it is a good teaching tool for those that have yet to dive into CSS based navigation lists (and you should get into it, as CSS is designed for such things).
For those that want a huge assortment of samples, code and really cool lessons, check out Stu Nicholls’ CSS Play. This site is an amazing reference for learning the art of cross-browser compatible CSS. There are menus, layouts and much more available to learn from and even use, in many cases without even a link back to him, though it is always a good idea to give credit where credit is due.
So if you are in need of a little CSS learning fix, hit these references up. You will enjoy them and, in the case of Stu Nicholls, may even make them part of your normal daily web development toolset.
Sometimes I wish that Microsoft were something tangible that I could grab hold of and slap 32 different ways from sideways as I watched it writhe in anguish begging for mercy. Why? Because it does that to me almost daily. And I am sure I am not alone.
Tonight my wife showed me that she had installed Opera 9.5 on her computer. I was very impressed with her technical savvy that she showed in doing this. So much so that I told her that Firefox has just come out with a new version and, since Firefox the previous has all but killed my wife’s computer, updating may be the order of the day so that she could use Firefox instead of Opera.
So I had her load up Firefox and attempt to update from within the application. And you know what? Two words: Epic. Fail.
Frick.
Firefox froze and took the entire system with it. Not wanting to watch Firefox think about dying on the spot, I decided to power down the computer and start afresh. Yes ladies and gentlemen, I killed the windows.
Apparently the registry was being written to/read from when Firefox decided to crap all over itself. And as such, shutting the computer down in the middle of that process freeze basically wiped out the registry. Thanks Microsoft for the awesome idea of the registry. I so love that.
So I got a blue screen of death (BSOD) on the boot. I tried logging in to the last known good configuration. Nothing. Safe mode? No dice. Safe mode with networking? Nada. Safe mode in any capacity? Pukage.
I was crapola.
So I hit up the Microsoft Support site and did a search for a corrupt registry hive file and found article #307545 - How to recover from a corrupted registry that prevents Windows XP from starting. Remarkably, this article saved my hide.
Now I will be the first to admit that Microsoft is the bane of all even semi-smart people’s existence. I hate them with a passion that is unrivaled in normal circumstances. But tonight… well, tonight I actually applaud them.
There knowledge base article actually worked almost to the letter the way they said it would. I had to guess a couple of times, but the guesses were logical and easy to make based on what I was seeing. In about an hour and half’s time I was able to get my wife’s computer back on its feet and ready to be (ab)used once again.
So tonight, for the briefest of moments, I thank you Microsoft for putting out a halfway decent article on getting a Windows XP machine back up and running after your stupid registry hive files go belly up at a time when they shouldn’t have. You actually earned your $32,000,000,000,000,000,000,000 USD tonight, in my book.
A few weeks ago I was toying with XML. I had a few minutes to spend teaching myself so I decided that I would use that to work on something that I desperately need to work on. XML.
And what better way to learn a little XML stuff that on the Amber Alert system? So I began my journey.
Continue reading »
As much as I loved my Chocolate Ocean theme that I developed a few months ago…
… I thought it was time for a change. So I changed.
My theme that is.
Continue reading »
I ran into something today that made me stop and think that I am not really as smart as I want or need to be. It is something that is easily overlooked for most people but should always be a priority for bloggers or developer’s that make blog themes.
Worse yet, it is something that I have ran across before but completely forgot about since.
Continue reading »