I'm down this rabbit hole.

My business debit card has expired.  I have purposefully allowed this to happen so that I can close the account.  The services that are still attached to it and that I want to keep alive need to be moved to a new account.  One of those services is web hosting.

I noticed that of all of the expenses, web hosting is the largest one.  I spend $25 per month for one virtual server at Rackspace, while simultaneously spending $20 per month for three virtual servers at Digital Ocean.  One of the Digital Ocean servers sits empty, waiting for the transfer of content from the server at Rackspace.

So, to save $300 per year, I've decided to migrate all of my personal web content to the preallocated Digital Ocean server.  Last night.

Server Configuration

The new server is an Ubuntu LTS server.  I'm running all of my web sites (there are 97 of them) via Nginx on this server, whereas I was running Apache on the old server.  There is an exchange here of configurability for performance.  Apache is easier to configure, and Nginx is less resource-hungry.

Nginx uses fastcgi to route requests to php-fpm.  I've configured fastcgi to use TCP instead of unix sockets because, counterintuitively to me, TCP is faster and more reliable.

Incoming requests first hit HAProxy, which proxies the request to Varnish, which proxies the request to Nginx.  Varnish caches built pages only on asymptomatic.net, since I haven't taken the time to see what other domains could benefit from a caching proxy -- there are probably many.  The manual expiry of cached pages is quite complex, even on this domain.

I have managed to configure HAProxy to failover to proxying directly from Nginx in the case that Varnish doesn't answer.  This would have come in handy last night when Varnish decided to vanish for about 7 hours.  Routing directly to Nginx is not ideal, but it's better than all of the sites going down.

What I'm looking for now is a simple monitoring tool to ensure/alert that any of these services go down.  Pingdom will alert me when the site itself isn't available, but it will not tell me when Varnish isn't doing its job properly and HAProxy is circumventing it.

Remaining Migration

Of the 97 domains hosted on the server, there are a handful that still need to move.  Some of them are simply just other domains that need re-pointed to this server.  Others are more complicated node-based services that I will need to configure directly in HAProxy so as not to need to use an odd port for the service.

I am pleased with the configuration of the environment so far.  Apart from the mysterious Varnish outage, things are running smoothly.  I have bumped the available memory on the server to increase the grease between rubbing components.

What I might consider, as an experiment, is using Varnish's file storage backend instead of the malloc backend.  Since Digital Ocean's servers are all using SSDs, write speeds should be reasonably quick.  If reads are likewise speedy, there might not be a lot of difference between malloc and file based stores, and I'd be able to keep more cache alive.  This would allow me to use the cache on more domains, since memory is in limited supply, whereas disk space is more easily obtained.

Soccer is upon us again, and I'm coaching Abby's team.  This year, I've decided to sign up as head coach, which is a big step in my head, but probably not much more than I've done in previous seasons.  Conceptually, I wanted to be able to run practices and games in a way I see fit.  It's not that prior coaches didn't do a good job - we took the field - just that I have some changes in mind for how to run things.

This season we'll be using incentives for the players on the field.  I've acquired a bunch of little soccer ball patches to give out to the kids that perform well at practice and games.  I hope this will give them some incentive to do their best, whereas in seasons past, I feel like they haven't all given their best effort for lack of it feeling like it meant much to participate.

I'm slowly constructing a training schedule.  I think this is one of the more important changes this year.  With only three training sessions before our first game, we need to fit in some quality training time.  And that's a distinction I'd like to make on the first day: We're running training sessions not practices.  Practice is what you do when you repeat the same things every day to get better.  We won't practice every day, so I'm going to be doing something else.  I'll show the players what they can practice at home, but focus our training on how players should work together during the game.

By our first practice I will have our team mantra finalized.  I know what I want it to consist of, but not how to shorten it into something memorable and effective.  Essentially, the idea is that every player is playing the game mentally all the time.  On the field, you need to be thinking about what you're going to do with the ball even when you don't have it, so that when you get the ball, you don't need to think.  I think this will effectively address some issues we've had in the past like players kicking the ball away and multiple players running to the ball instead of just one.  It should keep players looking up at their surroundings and engaged during the entire game, not just when they have the ball.

This season I want to encourage the team to become a passing team, with as many one-touch passes as possible.  I want them to grow to intuit passing routes, which I think will be my teaching focus for the season.

I'm currently planning on breaking up my three training days into basics, attack, and defense days.  Our first training session will help us get a feel for the players' abilities and how we can position them on the field during a game.  We'll drill heads-up dribbling, sprinting to the ball, shooting, down-field passing, and throw-ins.  We'll also need to identify a keeper, which always seems difficult.  I don't think I want to let the slowest kid volunteer for keeper like we have in seasons past, at least, not for entire games at a time.

One of the more challenging aspects of coaching is finding succinct ways to convey information to the players, but the same thing can be said, and is probably even more true, of conveying information to parents via email!  If I don't reduce the message to its essence, they don't bother trying to understand.

We're already seeing how fall soccer is an extra sport to keep some kids warm between other fall sports, which is kind of depressing.  Some players won't be able to attend soccer matches because they've got practices for other sports at the same time.  Why did they sign up for both?  I don't think it's fair to the kids that show up every game to have to sub out for kids that show up when they feel like it.  I realize this is just rec league soccer, but for some kids (like Abby), it's all they do, and these multi-sport kids that don't really care about soccer are taking away from those kids.  I'm sure I need a different perspective on this, because it simply seems inequitable.

I got a set of 6-foot goals to use at our training, and a set of pinnies to break the team into three small groups for some of our drills.  We've got balls from the league.  I need to check if we have cones or not, which we'll need on our first day.  Apart from the cones, I think we're good for equipment.

Here's looking forward to a good first session.

This year I've bought into a season ticket package with some other guys to go to Philadelphia Eagles football games.  I went to a couple of games last year with them, and it was a good time, so I was enthusiastic when they offered the opportunity to join them for the rest of the season.

As part of the season ticket package, there are tickets to home pre-season games.  Typically these games are on Thursday night, which makes them difficult to schedule to attend.  Combined with the fact that the big name players don't play, there isn't a big demand in our group for these tickets.  As a result, I requested all four of our tickets for the Jets game, and took Berta and the kids.

The game was entertaining.  It wasn't as busy as a regular season game by a long shot, but it was busier than any pre-season game I'd been to before.  I would say that that stadium wasn't quite half full.

I tried to convey all of this to the kids while we were heading into the stadium.  We took the train, which was probably a mistake, but always seems so convenient to Berta's work to get in and out of the city, which I really hate driving to myself.  Abby was amazed at the number of people that were in the subway with us headed toward the game.  I told her that it was just one subway train, and it that there would be many more people at the game itself.  She seemed impressed.

It's hard to convey the difference in atmosphere between pre-season and regular season games.  There isn't the same sense of urgency. The same energy isn't there.  The pageantry remains, but the crowd isn't as enthusiastic.  It's an odd thing.

Riley spent most of the game complaining about how he doesn't like football, "It's boring."  Instead of watching Philadelphia play the Jets, he watched the jets on approach to Philadelphia International Airport.  There were 25 that he counted.

I also wanted this to be an opportunity for Berta to see what the games were like to attend, since I'd be doing this every other week or so during the season.  There are some things that are a pain, like how you can't bring a purse into the stadium unless it's completely transparent.  The game itself, if you like football, balances a lot of that out.  The food prices are insane at the stadium, though.  I don't care how good you think Chickie and Pete's Crabby Fries are, that's too much to pay.  On regular games, we usually just grab a sandwich outside the stadium and pack it into the game in a clear bag.  Anyway, the game isn't always so expensive to attend.

There was a bit of a bother getting home since I hadn't taken into account the returning train schedule, and we got the kids to be a little later than we'd like.  Abby and Riley both commented about how they disliked the city -- it was dirty, polluted, full of weird people...  I was a bit sad that I have yet to convey the wonder (this is not the best word, but good enough for now) that I see in the city to the kids.  Maybe they will appreciate it more when they're older. 

Still, the evening was entertaining, and we survived.  Now the kids have been to a pro football game.  Hopefully they'll look back on it as the positive experience I was hoping for.

It's odd how memory works, isn't it?

When the kids started school this week, I was trying to remember my classes and teachers from 8th grade.  I didn't expect to remember the teachers' names, but I expected to remember what courses I had; like, what I learned in math that year.  I could not.

I do remember 9th grade math, which was geometry.  I had class in the same room as my homeroom for 9th grade.  The name of the teacher is on the tip of my tongue.  I thought the guy was pretty cool until I had him for math, but he was just another math teacher, really.  But I don't remember 8th grade math at all.

10th-12th grade math classes I remember quite well.  I remember only an impression of 7th grade geography, where I'm waking up from a nap next to the persistently closed blinds in the dark classroom where Mr. Bates only ever showed slides.  I remember home economics, and ironing material to make tote bags, and being partnered with that popular girl who wore a new Swatch to class practically every week.  I remember the TRS-80's in the special education room, my 8th grade homeroom.  I remember my cousin, immigrated from another elementary school, sitting near me in my 7th grade homeroom, and her incredulity at my academic attention and proficiency, unsurprisingly not correlating the two.

During all of this reminiscing I had a flash of Indian Acres, a place where we used to spend summer weekends with my grandparents near the bay.  I remembered harvesting the peanut plants near the shed, playing rummy under the yellow porch light that didn't attract as many mosquitos, buying the complete set of Hardy Boys books at a flea market in the teepee-shaped community center, launching the boat at the dock, diving in the public dump for salvaged refrigerator magnets.  The details of these things exist in varied clarity in my mind.

But I can't remember 8th grade math class.

Clearly, I learned math that year, or I would not have progressed.  Still, it's somewhat distressing that I don't even remember the course.  Was it pre-algebra?  Was it algebra?  Did we do something else that year?  I simply don't recall.

What if life consisted only of what memories we have (doesn't it?), and we spent our lives struggling to hold on to what memories we meant to keep?  Am I missing something significant in 8th grade math?  What am I missing from my memory that can't similar be indexed by the mere fact that I know I experienced 8th grade math in the course of my education?

It was just a year ago that I was writing about closing down my consulting business and joining ownCloud as an employee.  And now I've moved on to new digs.

Moving From ownCloud

ownCloud was a great opportunity.  I really love open source software - software that is written and distributed with the source code and the ability to change and re-release that source code.  I've been a contributor to open source projects for a long while, and have even started my own.  So the opportunity to get paid to work on such a project was a great find for me.

There were several things that attracted me to my job at ownCloud.  First, it was a product, not project work.  Rather than working on a new project for a new client every few months, this was a single product to concentrate on iterating and getting right.  Second, the daily schedule was pretty free to work how I pleased, as long as I met promised schedules and got done the work I was supposed to do.

In all, I was very pleased with my time at ownCloud.  The people there were nice to work with, and I was exposed to a different aspect of software company.

That said, toward the end, I felt that my experience was being wasted by not allowing me to contribute in significant ways to the fundamental design of the product. I felt that prioritizing some fundamental changes was more important than the things I was working on, and after sharing these thoughts, although they became more solidified in my mind, they were not the priority of the company for me to work on.  That was ok, and I understood it, but I wanted more.

I think I simply had a bad week at ownCloud, and in mentioning this in passing to a friend, I managed to discover CoverMyMeds.

What is CoverMyMeds?

The simplest explanation is long, merely because if you haven't had the problem, you might have no idea that it needs solving.

Insurance companies sometimes require doctors to specify reasons or meet certain other treatment prerequisites for prescribing certain drugs to their patients before they will approve payment on that drug.  To inform them of these prerequisites, the doctor must fill out a form which is usually specific to both the insurer and drug with that information for that patient, and send it to the insurance company.  The thing is, doctors don't keep track of this stuff, and patients usually don't find out they need it until the pharmacy, trying to get money from the patient's insurance company, gets denied their claim.

CoverMyMeds provides a system that is free to pharmacies, doctors, and patients to easily obtain the correct forms, fill them out, and submit them to where they need to go.

Interviewing at CoverMyMeds

My interview at CoverMyMeds was the most intense interview I've ever enjoyed.  Mind you, I've been at some lousy interviews - Vanguard comes to mind, ugh, chills - but half way in, after meeting the people, I could tell that they were places that I'd never want to work.  With CoverMyMeds, things were a little different.

My friend told me that my first interview would be a pretty casual "get to know you" interview with no technical questions.  But I found out quickly that they were fast-tracking things to try to hire candidates promptly.  While the technical questions surprised me, they weren't crazy; I was simply ill-prepared to answer those types of questions.  Fortunately, I apparently know enough unprepared to succeed at answering those.

I had a second Skype interview where I solved some basic coding questions.  These were deceptively simple problems, and being a little nervous at someone watching me work, I think I flubbed a couple of things that I should have easily passed.  Still, I did well there.

For my last interview component, the company flew me to their office in Columbus, Ohio.  I spent the day building a small application based on their specifications, then presenting the app, my code, and an explanation for my decisions to a group of their developers.  I answered questions about what I did, and even some impromptu javascript puzzles.

A lot of the interview was oriented toward seeing if I would be a good corporate "fit" within the team.  I really liked the people I met.  They all seemed like savvy developers, and had similar extracurricular interests to mine.  I think we got along very well.

Ultimately, I think I did well at the interview, because they offered me the job!

Working at CoverMyMeds

I've been trying to explain to friends and former co-workers what it's like to work at CoverMyMeds, and always feel like I'm coming up short.  I can summarize by saying that it's a pretty great place to work, but there are some specifics.

For one thing, they've figured out management.  I suppose this has the potential to change because such a thing is fragile, but right now, management is pretty nice.  Unlike many places that "talk the talk, but don't walk the walk", this place "walks the walk".  When they say that the company believes in the well-being and happiness of its employees, they actually mean it.  I have some silly examples, but trust me - it's pretty great and unique.

Another thing is that the development process is thought out.  It's certainly not dogma; there aren't rules for every little thing.  There are enough rules to make sure everything gets done smoothly, and make sure that everyone knows how things are supposed to be done and what's expected.  And that's it.  It sounds pretty simple, but so many places lack this discipline with their development process, it's surprising and refreshing to see it written, in place, and working.

Something that attracted me to CoverMyMeds was the chance to learn new technology.  It's a bit complicated to explain, but suffice to say that CoverMyMeds is primarily a Ruby shop.  Learning Ruby has always been on my professional to-do list.  It was difficult to pass up the opportunity to leverage the skills I have as a PHP developer while learning Ruby and being paid to do so.

CoverMyMeds does Pair Programming, which is an activity where two developers are "paired" to work together on a particular feature or project.  It's very hard to explain to family and friends how this really works, but it ultimately results in me being tethered by headset to a guy in Florida all day.  It sounds potentially awful, and I'm sure my pair would agree that it has its downsides, too, but all-in-all, it's been pretty nice to have someone to help me work through things, especially while I've been learning the ropes.

Our current project is leading us both into exciting new development waters, and when it's done, it'll be pretty fulfilling to see it in use.

What's Next?

I hadn't really thought about it before starting at CoverMyMeds, so it's both somewhat strange and oddly telling that I'm considering it now.  "What will I do next?"

Well, if I had thought of the question at ownCloud, I still might be thinking of doing something new.  And with my consulting business, I was struggling to do new things all the time just to keep money coming in.  And at Rock River Star, what I wanted to do next and what was available to do next were never going to be the same thing.

So now, when I think of the question, I don't think that I want there to be a "next".  That seems pretty good to me, and I hope that next year, when I'm writing another "State of the Job"-type post, I still feel the same way.