Setting Up Let’s Encrypt on an Azure App Service

Once I had my blog ported over to WordPress running as an Azure App Service, I knew I needed to figure out how to secure my site, both because I wouldn’t want to be logging into it, randomized unique password or not, and because I wanted to be a good web citizen and secure all the things.

I saw that my pal Jeramiah had used Let’s Encrypt to secure his site, so I did some Googling, and asked him if he’d used the Azure extension I’d seen mentioned in a few blog posts, and he confirmed he had.

I read a few guides on getting it done, and while I had a few issues along the way, I finally got everything working. In an effort to save people from the starting and stopping and Googling that I had to go through while working through it, I decided to document the entire process from start to finish on a fresh blog.

You’re going to need an Azure Storage Account

Once you’re logged into the Azure portal, look for Storage accounts on the left-hand menu.

Azure Storage Account 1

As you can see here, I have no storage accounts. Click Create storage account.

Azure Storage Account 2

You’ll need to fill out and select some options here. I like to name every account or aspect of an App Service based on the overall name of the App Service, so I went with geekfoodblog.

I left Resource Manager as the default, selected general purpose v1, located in East US, and selected Geo-redundant storage (GRS). That may be overkill for my needs, but based on the storage costs for this blog last month and the amount of MSDN credit I have, it’s moot.

I believe Secure transfer required was Disabled by default, and I left it that way. If you have more than one subscription, you can select that here.

I did choose to drop it into the existing geekfoodblog Resource Group, since I had already deployed WordPress as an App Service before setting up Let’s Encrypt.

I did not choose to configure  virtual networks, nor did I pin this account to my dashboard, and with that, I clicked Create.

Azure Storage Account 3

Click on Access keys and copy your primary Connection String into a secure note somewhere for later use.

Azure Storage Account 4

Now you need a Service Account (or App Registration)

You may see this referred to elsewhere as a Service Principal. Azure calls it an App Registration. So click on Azure Active Directory, then App registrations, and then New application registration.

Azure Service Principal 1

You can see how I named mine. You’ll make use of auto-complete later, so using a few memorable letters as a prefix helps.

Also, as noted below, the Sign-on URL doesn’t matter in the sense that it doesn’t have to be something you own or are working with, but it does has to be something that is recognized as a legitimate URL.

Azure Service Principal 2

Now you’ll need to create a client secret or key.  Click Settings, then Keys.

Azure Service Principal 3

Give your key a description – I used letsencrypt, and I selected Never expires as the duration. That’s probably terrible, but it’s a huge key, so sue me.

When you click Save, you will be warned to copy the key value, as you won’t be able to retrieve it later. Stick that in the same secure note with your Storage Account connection string from above.

Azure Service Principal 4

You’ll also want to copy and paste the Client ID of your Service Account / App Registration.

As you can see below, and as you’ll notice in the screenshot I saved with the Client ID error I left in, the Client ID is not what you named the App Registration, but rather the Application ID.

You can copy and paste this into your secure note, or you can go grab it later as I did.

App Registration App-Client ID

Time to assign permissions for your Resource Group to your Service Account

Now you need to make sure your Service Account has permissions to your Resource Group, in particular so it can access the Storage Account you created above.

Click on Resource groups, then on the Resource Group of which your Storage Account is a member.

Azure Service Principal 5

Click on Access control (IAM), then click Add. For Role, select Contributor.

Start entering the name of your Service Account in the Select field, and select it, then click Save.

Azure Service Principal 6

Azure Service Principal 7

Now let’s install the Let’s Encrypt Extension

But first, so you can avoid an issue I noticed when I first set this up, let’s ensure your App Service is configured to always be on.

Click on App Services, then click on your App Service.

Azure Extension 1

Now click on Application Settings, and scroll down to Always On and make sure it is set to On.

Mine was not for some reason, and I noticed an error at one point.

Azure Extension 2

Now click on Extensions, then Add.

Azure Extension 3

Look for Azure Let’s Encrypt by SJKP. Click on it, then OK to accept legal terms, then OK again.

Azure Extension 4

Before proceeding, to help you avoid an issue I’ll show with a screenshot later, go ahead and restart your App Service.

Scroll up to click Overview, then click Restart. Then scroll back to and click Extensions.

Restart Service

Click on Azure Let’s Encrypt, then click Browse.

Azure Extension 5

Azure Extension 6

If you didn’t restart your App Service, you might get this error below.

Azure Extension 7

Now fill out the Let’s Encrypt Authentication Settings

First you’ll enter your Tenant URL, which will be unique to your Azure tenant.

You’ll then add your Azure SubscriptionID – also unique to you.

Next, for ClientID, you’ll enter the Application ID of the Service Account / App Registration you created above. Did you copy and paste that into your secure note? If not, you can find it under Azure Active Directory > App Registrations > Name of your Service Account.

For ClientSecret, enter the Secret / Key from your Service Account / App Registration.

Enter your ResourceGroupName and ServicePlanResourceGroupName – which for me are the same thing.

Be sure to check Update Application Settings, as this is required for the web job that will renew the certificate later.

Azure Extension 8

At this point, assuming you already have your hostnames configured, you should see something similar to what I did below. So click Next.

Azure Extension 9 Azure Extension 10

Select the hostname, enter your email address, and click Request and Install certificate. 

I’d already done this once before, so I was fairly sure it would work, so I didn’t bother checking the UseStaging box.

Azure Extension 11

Now you’ll need to add the SSL binding to your Azure-hosted domain. So go to App Services Your App ServiceCustom Domains.

While you’re here, if you haven’t already done it, switch HTTPS Only to On.

Scroll down and click Add binding next to your domain

Azure Extension 12

Select your custom domain under Hostname. Select the new SSL certificate under Certificate. Click Add Binding.

Azure Extension 13

Time for some Azure WebJobs goodness

If you stopped right now, your site would be secured until the Let’s Encrypt SSL certificate expired in 3 months. Let’s ensure that doesn’t happen by connecting your Let’s Encrypt WebJob to the Azure Storage Account you created above.

Go to App Services Your App ServiceApplication Settings.

Scroll down to Connection Strings and create AzureWebJobsDashboard and AzureWebJobsStorage.

Both of these should have a value of, you guessed it, the Connection String you copied from your Azure Storage Account above.

Azure Extension 14

You can confirm your WebJob is running by going to App Services Your App ServiceWebJobs

Azure Extension 15

And once you’ve done all this, fire up your web browser, go to your custom domain, and check your your shiny new Let’s Encrypt SSL certificate.

Azure Extension 16

WordPress as an App Service on Azure

I’ve blogged on the WordPress platform for years, starting way, way back when I had what I thought of at the time as a shell account at Pair Networks. Since then I’ve installed and run WordPress on other web-hosting accounts, as well as on virtual private servers and, for a short period of time, even on a spare Linux box under the desk in my office. I’ve spent most of my career doing Windows system administration and a goodly bit of it using a Mac as my primary desktop/laptop computer, but I learned just enough Linux to install and keep Apache, PHP, MySQL, and WordPress running. At some point I grew tired of caring and feeding for WordPress itself, so I just imported my blogs into WordPress.com, paid for domain mapping and their “no ads” service, and let the folks at Automattic worry about it.

Will This Be Hard? No.

My first thought about running WordPress on Azure was that I would rather not go back to managing WordPress the old fashioned way involving managing the entire stack from the OS (Linux or Windows) on up. Turns out, as Jeramiah alluded to in his recent post, I don’t have to. There’s certainly more opportunity (and need, especially since I wanted to make my Azure-hosted blog secure) to fiddle with nerd knobs running an Azure App Service, but when it comes to getting WordPress up and running, it took about the same amount of time on Azure as it did at WordPress.com. Want to see how easy it was? Let’s build another one together.

1. Log into the Azure Portal and click on App Services, then click Add.

0718 Azure Add App Service

2. You may be tempted to select one of the WordPress options you see right away. Resist that urge, unless of course you want to run WordPress on Linux or something else.

0718 Azure App Service Search

3. Instead, type WordPress into the search and hit enter. Select just plain WordPress as shown below, then click Create.

0718 Azure Just Plain WordPress

4. This next step is important for a few reasons. First, whatever App name you choose here will become your hostname in the domain azurewebsites.net. Second, you will choose whether to create a new resource group or (if you have one), use an existing one. Most importantly, and it may not be obvious at this step (it wasn’t to me), you’re choosing whether you want to run and pay for a separate database service to run MySQL. I went that route at first, but after conferring with Jeramiah, I decided I’d rather save the money/credit and just run MySQL inside the App Service plan. I’ve included the disclaimer Azure shows you below as well.

Azure App Service Options0718 Azure DB Disclaimer

5. Click Create. I chose to pin my new App Service to my dashboard.

So five steps (maybe a couple more total clicks) to deploy. It takes Azure a minute or two to deploy the new App Service, and once it’s finished, it is fully live, as shown here:

Azure WordPress Setup

And just a minute or two after filling out the basic info for the WordPress Setup, I had a working install up and running, and even prompting me to update to the latest version.

Azure New WordPress

Back in the Azure Portal, I was presented with a nice data-rich view of my new App Service, along with lots of options, some of which I’ll go into when I detail how I used Let’s Encrypt to secure my new Azure blog.

Azure App Service Dashboard

 

And once I finished taking the screenshots I needed for this post, deleting the App Service was just as easy as creating it. Just click Delete, confirm by typing the App Service name, and click Delete again.

 

Azure Delete App Service

So Why Do This?

That’s a fair question. As I mentioned in my previous post, this blog was being neglected over at WordPress.com, but I could have simply fired up MarsEdit and kept posting to it there. But I want to learn more about Microsoft Azure, maybe get outside my comfort zone a little bit, and I figure one way to encourage me to do that is to port this blog over and set myself a challenge to document the experience. So that’s what I’m doing.

If I didn’t have an MSDN subscription with a monthly Azure credit, would I pay to host my blog here full time? I don’t know – maybe, maybe not. But I do, so I am. I figure hosting my blog is the least interesting thing I can do in Azure, but it’s a start.

If you have suggestions for other stuff I can try in Azure, let me know via Twitter, where I’m @mikestanley

 

 

 

Climbing Back into the Saddle

I love to write. Heck, my email address at work is poet@nospamplz.edu. And yet, somehow, I haven’t written a single blog post for more than a year and a half. The reasons why don’t matter. What does matter is that I’m tired of not writing, and I recently received what was almost certainly an unintentional kick in the behind from my buddy Jeramiah Dooley. Jeramiah published a post entitled “Who Needs Some Lab Gear” on his blog. I was intrigued that Jeramiah, a guy I met when my former employer was considering the vBlock, had started a new job at Microsoft in the Azure engineering group, and was divesting himself of a ton of “home” lab gear and moving his blog over to Azure.

That got me to thinking. I don’t really have a ton of lab gear… in fact, what little I do have, shown here, hasn’t been powered on since we bought our current house 13 months ago, so I should probably get rid of it as well.

Home Lab

What I do have, however, is an MSDN subscription with a healthy monthly Azure credit. I’d played with Azure a bit over the years, spinning up a virtual machine for a few hours until I did the math and realized I couldn’t keep it running all month, but that’s about it. I’ve been using it to test Microsoft Intune for a project at work recently, but when Jeramiah mentioned he “moved everything over to an Azure App Service, so no need for servers at all,” my curiosity was piqued, and I figured I could at least try to do the same thing.

Azure’s Pretty Cool, and Easy to Work With

I’ll detail this in another post, but it turns out, getting something up and running in Azure is crazy easy. I even messed up a few times, selected the wrong option without considering the consequences, but that wasn’t a big deal – I just deleted the App Service and started over. Once I decided what I wanted, it took me far less time to get the basic service up and running than it did for me to decide which photo I wanted to use as the header image for this blog. Here’s my current Azure dashboard:

Azure Dashboard

Since this was a learning experience for me, my next post will detail the process of getting a blog (on WordPress, in my case) up and running in Azure. For now, here’s a a meme:

Blogging Azure Style

Non-IT Training for IT Staff – It Matters

IT Training (the norm)

I’ve worked in IT for more than 20 years, with most of that being in higher education at the University of Tennessee. We’re a state-funded university, and the training “budget” (when there even was one) in the various groups I’ve worked for has never been what I’d call large, and for many years I would have said it was non-existent.

Still, I’ve been sent to numerous IT training classes or conferences over the years – most funded by my group/unit, with a couple funded by a vendor. While working on the HelpDesk, I volunteered to attend, along with a few other people, evening classes over a six month period of time, to get my MCSE, way back in the Windows NT 4 days. Since then, I’ve attended Microsoft, Citrix, and VMware classes, and attended a handful of conferences, including MMS, Synergy, TechEd, Ignite, and Dell World.

I worked for various incarnations of the central IT organization for UT Knoxville for 17 years. During that time, I received the bulk of the IT training I refer to above, and I’m grateful for it. With the possible exception of a couple of Microsoft courses delivered by professional trainers in the early 2000’s whose primary skill seemed to be the ability to read the official courseware out loud, every class and conference I’ve ever attended has made me a better IT Administrator.

Non-IT or “Soft Skills” Training (not the norm)

I work at a university, so it wouldn’t be fair to say that I’ve never received any non-IT training until recently. But I believe choosing to work on a second BA in Creative Writing using my (then) university-granted fee waiver benefit was fundamentally different from what I’m about to write about.

Since 2013, I’ve worked for the UT Institute of Agriculture, and my boss, our CIO, places a high value on developing his employees not to be just better technologists, but better leaders. Given that, he asked me last year if I’d like to attend the Dale Carnegie Course. I knew he’d sent a couple of my coworkers through the course before I was hired, so I said sure.

I remember thinking in vague terms that this course, as many people do, was mostly about  helping people become better public speakers. I’ll write more specifically about the course someday, but let’s just say calling the Dale Carnegie Course a public speaking class would be like calling Walt Disney World a place with a few neat rides. Sure, it is, but it’s so much more than that.

The Dale Carnegie Course may not have taught me anything directly related to doing my IT job in a technical sense, but I can say without hesitation that it didn’t just help me become a better employee and a better member of my group – it’s helped me be a better person. After missing a session due to some work travel, I recently made up that last session and graduated from the course.

What’s so awesome about working for the Institute of Agriculture and UT Extension is that this sort of training and professional development is promoted throughout our organization. My boss paid for some of this course from our group’s budget, but he didn’t have to pay for the whole thing because I applied for and received the Lloyd and Nettie Downen Endowment Fund Leadership Enhancement Award. That $1,000 award covered about 60% of the cost of the course.  I continue to be impressed with how much of an investment the leadership at the Institute places in developing its employees.

My Advice

If you’re a manager of IT people, consider investigating non-IT training that may benefit your employees. If you’re an individual contributor, look for a course (I can highly recommend the Dale Carnegie Course) and see if your boss might be willing to fund it. We may all work in IT, but so much of what we accomplish in our jobs is impacted by and dependent on the relationships we have at work – with coworkers, partners, leadership, customers. Learning about new technologies can pay off well. Learning softer skills can pay off even more.

 

 

 

Carnegie-Certificate

I’m actually more proud of this certificate than I am of either of the two diplomas I have for my degrees.

 

Carnegie-Grads

My coworker, Daniel Hinton, and I attended the Dale Carnegie Course together.

 

A [very late] Update on Carpal Tunnel

Last year I posted that I had to take an unplanned break from blogging. That break lasted longer than I expected on this blog, although I did manage to post periodically on my food blog, Geek Food Critic.

Well, I’m back with a short update on what happened, how I dealt with the continued issue of RSI/carpal tunnel syndrome, and word that I’m once again participating in the 30 day blog challenge #vDM30in30.

The Medical/Body Side of Things

As of last December, I was waiting to see a specialist. I managed to finally get an appointment with him in late December. I saw an orthopedic surgeon who had actually performed an operation on my wife’s elbow, and she liked him a lot. I did too, although I didn’t really enjoy the nearly two hour delay in being seen the day of my appointment. He was apologetic, however, and I could tell the delay was due to how much time he spends with each patient.

I described my pain, numbness, and tingling to him and he did several manipulations and tests of my hand and fingers. I ended up getting a cortisone (I think) shot that day, and let me tell you, that was both terrifying, as someone who isn’t fond of needles, and oddly pain-free, thanks to some sort of magic cold spray the nurse used on my wrist. After a day or two the shot kicked in and it was like magic. My hand and wrist felt better than it had in years. I kept using the various trackballs and even the Evoluent mouse my boss ordered for me, but figured I might be able to just go back to my usual Logitech mouse.

Wrong. The magic wore off three weeks to the day from getting the shot, and the pain was even worse than before.

So I went to see the doctor again, but first I saw another doctor to have some sort of strange electro-shock torture test done on my hands and arms. I believe this test measured the time it took electrical impulses to travel up and down my arms and hands and fingers, and would have been required by my insurance company (oh how I love my health insurance company) before any possible surgery to address the carpal tunnel syndrome.

Except it turned out that the test was fine, mostly anyway, and showed no serious nerve damage. I asked the doc if that meant I’d just caught this early enough that none had occurred yet and he said that was possible. He didn’t recommend surgery, which was fine with me because I don’t want to be cut open if it isn’t absolutely necessary. So I got another cortisone shot, with an explanation that he really only recommended 2-3 of those in a calendar year, so hopefully this one would last longer.

And it has – right up until this week. That familiar pain and tingling is starting to come back, so I would imagine I have another trip to the doc and at least one needle in my wrist in my near future.

The Tech Side of Things

So what have I been using or avoiding in trying to deal with this issue over the last year? I ended up trying multiple trackballs, a Logitech trackpad, the Evoluent wireless mouse, and what I settled on for regular use at home and at work for a while was the Logitech M570.

Logitech_M570

 

At home I adapted to using the M570 for gaming for a few months, but I noticed some discomfort after an hour or so, so I’ve mostly stopped gaming on the PC on a regular basis. That meant I used either my MacBook Pro or my iPad Pro at home, and over the last few months, I’ve used the iPad Pro almost exclusively. It’s easier on my hands and it’s plenty powerful enough to do what I need to do. I’ll be finishing this blogpost later tonight on the iPad, in fact.

At work, I use the M570 almost all the time when I’m using my MacBook Pro at my sit/stand desk. Recently, for at least 1-3 hours every day I have to work with a PC to work with networking and security equipment, and while doing that I use my old Logitech mouse, but I try to grip it lightly and take my hand off it and use the keyboard as much as possible. Typing that out makes me realize I need to just put in an order for another M570 for the PC – thankfully they’re cheap.

I use my MacBook Pro rarely in meetings at work – it’s a 15″ beast, after all. For most meetings I use my 9.7″ iPad Pro with the Logitech Create case.

Supplements I’m Taking

Something I hadn’t tried by last December but did start by the time I saw the ortho doc was taking a couple of supplements based on recommendations from friends who had dealt with carpal tunnel syndrome. I’ve taken Tumeric in capsule form twice daily for the last 11 months, as well as a B12 vitamin every day. I’m not sure if they’ve helped, but from what I’ve read they both help with inflammation.

So What Comes Next?

I don’t know, but I’m going to start using an M570 on my PC at work and hope that reduces the irritation of using a normal mouse as much as I’ve been doing lately. I’m also very close to going all iPad all the time at home, given my disappointment with Apple’s latest and very late new MacBook Pros.

If that helps, great. If it doesn’t, I’ll see my ortho doc again and see how another cortisone shot works. I’d like to think that surgery isn’t in my future, but the truth is I use my hands all day every day to make my living, and I’m not sure what else I can do to lessen the impact of all of the repetitive and stressful movements I make doing my job.

An Unplanned Break from Blogging

So this will be even shorter than the one I’d spent a painful half hour typing before Squarespace decided to eat the post.

I completely failed in my commitment to keep up with and successfully complete the #vDM30in30 blog challenge. 

I did that for two primary reasons:

  1. We decided we needed a new, larger vehicle in November. As is normal for me, I obsessively researched the purchase.  We settled on a Kia Sorento and bought it the Saturday after Thanksgiving. Most of the spare time I had in November was spent doing car research, and nearly all of that on my iPhone 6S Plus because of reason 2.
  2. In late October I developed carpal tunnel syndrome. I’m still dealing with it now, having seen my primary doctor and am currently waiting to see a specialist. 

I typed up a good bit of info about my experience so far in the post Squarespace ate (and that I, admittedly failed to save – won’t make that mistake again) so I’ll just post a pic and a few links to what I’m trying right now.

This is me with the 2 trackballs I’m using with my right hand these days.  Not pictured is the one I also use with my left hand.

I’m using the Kensington Orbit with my left hand when I feel like I need to give my right hand a break. It’s cheap and not the best feel/quality, but I had it in a box at home and it’s good enough, for now anyway, for my off hand.

For my left hand, I am alternating between the Logitech M570 for precision work and the Kensington SlimBlade for general use. I bought the Logitech right away when I started hurting because I didn’t want to wait on the purchase process at work, and I figured I would need one at home anyway. I’m currently borrowing the Kensington Slimblade from my boss for a few days because, as the most expensive of the three, I want to make sure it will do well by me before asking him to drop nearly $100 on it.

I’m also going to borrow an Evoluent vertical mouse from a colleague this week. I’ve heard good things about vertical mice. I have tried and will be returning an Anker vertical mouse. It’s too small for my hand and gets fairly crummy wireless reception.

More to come as I figure all this out

I’m planning to post more about this as the pain allows and as my experience with it grows. I’m going to try to experiment with blogging via dictation either via my Mac or iPhone. While I anticipate needing to do some layout and image adding by hand, just being able to save typing most of these words would have been very nice.

My First Terrible Experience at the Apple Store

I don’t like the term “fanboy” but I’ve had it applied to me many times due to my preference for and, admittedly, advocacy for Apple products among my friends and family. I switched to the Mac at work back in 2002 because our Mac person left, and within a few years, all of the computers I owned were Macs. I still prefer them today, although I work more with Windows on a regular basis, and I’ll admit that Windows 8 and Windows 10 appeal to me more than 2000/XP ever did.  All this is to say that I’m firmly grounded in the Apple ecosystem, both on the OS X and iOS side of the house.

I have always loved the Apple Store. I remember when ours opened up here in Knoxville, and it is still the only store I will gladly walk into at the mall. Over the years I’ve had occasion to visit the Apple Store a handful of times to have Macs repaired, and in one instance replaced – perhaps a story for another time highlighting how great Apple customer service can be. But today I want to briefly explain how bad the Apple Store experience can be, as I recently had reason to take my 5 days old iPhone 6S Plus in to be looked at.

PERSONALIZED SERVICE AT THE GENIUS BAR, RIGHT?

I made my appointment at the Genius Bar using the Apple Store app, for the first available slot, around lunchtime two days later. For the next couple of days I anticipated walking into the store for the same individual, personal attention I’d received numerous times over the years. I knew the Apple Store is crazy crowded these days, but surely, if I’m making an appointment, especially two days in advance, I’d receive the same type of customer service I always had. It is, after all, one of the things Apple has proven a real differentiator in its retail experience compared to its competitors over the years.

I couldn’t have been more wrong, although I wouldn’t realize that until about 10 minutes into my appointment. Because everything started out the same – check in with the traffic cop employee, who notifies someone via their headset that I’m here, then a helpful Genius showed up to escort me over, not the Genius Bar, as it was packed, but to the side of one of the tables across from the bar. And for a few glorious minutes, I received the kind of customer service and personalized attention Apple is famous for. Until it became obvious that my problem wasn’t going to be resolved in 10 minutes, and the Genius’ next appointment showed up.

If you have a problem that can be solved in under 10 minutes, you may still be able to walk out of the Apple Store feeling like you received targeted personal service. If, say, you have to restore your iPhone 6S Plus from an iCloud backup (15-20 minutes, minimum), what you will experience is a sort of frantic, start and stop, hold on while I check on this person kind of triaging that can probably be OK to terrible, but simply can’t be great.

For longer than an hour and a half, I was juggled along with at least 6 (I stopped trying to keep up with them) other Apple Customers by one frantic Genius moving between at least 2 locations. He’d come over to me for a minute or three, try something, start a process that could take another 10-30 minutes, then move onto another customer he was helping. And with up to 4 of us at any one time, the squeaky wheel got the grease, or in this case the attention of the Genius, as he seemed to be caught by the couple of customers he had over at the Genius Bar several times while I waited (not so) patiently over at the table where he’d started with me.

THE GENIUS DID THE BEST HE COULD WITHIN A BROKEN SYSTEM

Even though I was and continue to be extremely dissatisfied and disappointed with the experience I had at the Apple Store, I realized then an now that the individual Genius working with me was not to blame for the situation. He was polite and seemed genuinely interested in helping me, but he is working within a system that simply doesn’t afford him the ability to truly focus on an individual customer or problem they may be having for more than 10-15 minutes.

I worked on the HelpDesk early on in my IT career. I know how difficult and stressful it can be to help people who come to you with something that isn’t working. Imagine trying to do the HelpDesk job, but instead of answering the phone and dealing with one customer with a problem, if you don’t get the first customer issue fixed within 10 minutes, you have to start putting the first customer on hold at what is hopefully a natural pause point, to work with a second customer, then a third, and a fourth if necessary, all the while cycling between them. Every shift from customer to customer, problem to problem, must have an impact on efficiency, not to mention the inherent time to resolution increase while 1-3 customers sit there waiting to receive attention again.

Within the confines of this insane situation, the Genius did the best he could, and it was obvious either he’s naturally good at placating frustrated people, or Apple does a good job of training its employees to say the best words in this type of situation to try to mollify someone who, quite reasonably, is tired of being put off and juggled like a ball in a circus act. I don’t fault this young man for the environment in which he’s working and the decisions that someone above his pay grade made to turn the Genius Bar into a frantic, disjointed assembly line.

APPLE KNOWS HOW BAD THINGS CAN BE

The day after my terrible experience at the Apple Store, I received a request to fill out a survey for it. I did so with as much bluntness and perhaps a bit more brevity as you find in this post. I noted that yes, I was willing to discuss the matter further if Apple needed to reach out to me, and sure enough, a couple days later, I received a phone call from someone at the local Apple Store.

We had a good conversation. Not a happy conversation, but a good one. I explained my history with Apple and the Apple Store in particular, and how frustrating the recent experience was. More than anything, I tried to convey that the experience made me feel like I was just a box to be checked off. The guy I spoke with was very apologetic about that and said that wasn’t at all what Apple wanted, and I get that.

I told him that after thinking about how absolutely packed the Apple Store is nearly all the time, I don’t know if there’s a good way for Apple to solve this problem. They can’t just hire more Geniuses – there’s no room for them or additional customers. I suppose they could double or quadruple the size of the store (assuming they could get the space) but that may just be a bandaid. I suggested exclusive appointments of a longer nature by default, but realized that would necessarily extend the wait time for an appointment beyond the 2 days I had, and possibly as long as a week or more. The Apple employee noted that, especially given the critical role an iPhone and other Apple devices play in people’s lives, asking them to wait a week for an appointment would likely just result in even more walk-in customers who are unaware of or choose to ignore the appointment reservation system. He said they do their best to accommodate walk-ins, but increased wait times would make it even worse.

He asked if I was aware of AppleCare’s telephone support, and noted that many issues they help with in store can be resolved over the phone. I told him I was, but as someone with a lot of years in IT, I rarely called any type of support because for nearly anything phone support could help me with, I usually find those solutions myself on the web. I also noted the Genius had, perhaps later in the process than I’d preferred, punted and simply swapped my iPhone 6S Plus for a replacement unit – so assuming that was actually necessary, phone support would’ve done me no good.

I thanked him for calling and I’d say our conversation was cordial and wistfully hopeful, but grounded in the reality that today, Apple has so many customers, especially iPhone customers (as I was in this case), that there is only so much that can be done logistically to improve this situation.

Moments of Awesome in the Midst of Frustration

Even while I was growing more dissatisfied and frustrated with the situation at the Apple Store, I paid attention to what was going on around me. It’s not like I had anything else to do other than get hungrier as my lunch hour came and went, after all. I observed a couple of really great interactions that helped then and now to remind me that Apple really does try to do customer service better than anybody.

The first involved a customer I’ll call Extremely Rude Old Dude. This guy was mad because his iPhone 6 (maybe 6S) wasn’t sending email from his Gmail account. His tone and demeanor were really off-putting, so much so that just as an observer of the poor Genius doing his best to help him, I found myself wanting to just say, “Hey buddy, how about you dial back the attitude a bit and cut this kid some slack while he’s trying to help you?” Being raised in the South, however, I didn’t say that to one of my elders, although I did keep thinking it. Through it all, however, the Genius kept his happy Apple game face on, and never once reacted to the rudeness he was dealing with. And you know what? By the time Rude Old Guy left, his problem was solved, and he even had Reachability explained to him so he understood it was there to help him, not a problem with his iPhone. I’d like to think if this guy had been a customer of mine, I could’ve been as nice and patient as the Genius helping him.

The second involved a customer I’ll call Tech-Savvy Grandma. She walked in with a white plastic MacBook and said she was giving it to her grandson and would like to have it wiped and have a fresh OS put on it. I just checked and this model of MacBook was discontinued in 2011, and her’s could’ve been older than that. So there’s no way it was under active warranty. But this is where Apple really shines. The Genius (the one helping me, in fact) just nodded, asked her to confirm there was no data on the MacBook she needed, and went to work. He connected the MacBook to the Apple Store network via Ethernet (I assume they have to use USB/Thunderbolt adaptors for newer Mac laptops), did a NetBoot, and installed the latest version of OS X that supported that device. She left happy with service I know she couldn’t have gotten anywhere else.

Will It Get Any Better?

I don’t know. I kinda doubt it. If Apple had remained just a computer company, this wouldn’t have happened. Of course, Apple might not have survived or at least thrived as it has on the basis of the Mac and iPod alone, so who knows? But given the truly enormous customer base the iPhone has, not to mention the iPad, and continued growth on the Mac side while the rest of the PC industry declines, I don’t know that Apple can fix this problem. I just know I came away from it feeling reluctant to risk returning to the Apple Store to ask for help unless I had no other option, and that is not how I want to feel about it, and I know that’s not what Apple wants its usually very happy customers willing to pay a premium for its awesome products to feel.