The Paradox of Modern Web App Deployment

In the world of web development, such is the pace of innovation that barely a week goes by without an announcement of a new JavaScript framework. At the time of writing, there are more than 1,500 job descriptions in the U.S. on LinkedIn containing the word “AngularJS” and 750 containing the word “Ember.” Major tech companies have been at the vanguard of the single page app adoption curve, but increasingly, we’re now seeing large corporations embrace this trend be it Angular, Ember, or more recently React.

So, it comes as something of a surprise that while we’re seeing widespread adoption of modern web app development, there’s not more innovation when it comes to deployment of those web apps.

Indeed, the paradox of deploying these web apps is that most deployment options today are both overkill and inadequate at the same time. They’re overkill in terms of both price and complexity since many still entail provisioning an entire virtual machine for what amounts to mostly serving static content, and they’re inadequate with regards to the features endemic to this style of development. I would contend that this holds true whether one’s talking about deploying internally within an enterprise, or externally using a third party service.

For public deployment, Heroku remains a default choice for many web developers. However, while that choice made perfect sense 5 years ago when everyone was building end to end rails apps with Postgres backends, that’s increasingly not the situation for most web apps being built today. Indeed, with the availability of BaaS such as Firebase and Parse, as well as the proliferation of developer APIs for everything, a modern front-end developer can now build full-featured apps without ever needing to roll any server-side code of their own.

So, if we don’t need VMs for our web apps, what are the choices? Well, more enterprising developers can deploy to an AWS S3 bucket or Github Pages can be a good choice for quick public projects, and best of all it’s free. However, with Single Page Apps on S3 or Github Pages, there’s a common set of features that devs need that either aren’t available or come at a cost of increased complexity such as SEO snaphots, API proxy, OAuth, blue/green deployments, and more.

For public deployments, Aerobatic (the company where I’m a co-founder) and Divshot are legitimately focused on these web apps and provide many of those relevant features. For the enterprise though, what choice do they have?

Instead of VMs or scripts to provision infrastructure on AWS for these web apps, would it not be better to rethink the whole application deployment scenario entirely? Meteor is onto something when they imagine a multi-tenant deployment platform that can run inside an enterprise.

“Galaxy will be a product that the operations department at a large company might buy. It’ll be an enterprise-grade, multi-tenant hosting environment for Meteor apps. In other words, it’ll let you run a private, centrally controlled “meteor deploy”-like service for your company, on your own hardware. You’ll be able to manage how your apps are distributed across your datacenters, perform capacity planning, and enforce controls and policies that are appropriate to your organization. Google and Facebook have these tools — why shouldn’t your organization?”

However, this presumably would depend on a developer using the full Meteor stack and the reality is that most companies are a heterogeneous mashup of lots of different javascript, services, APIs, and databases. Seems that enterprises would want to have a deployment platform that can play well with whatever frameworks and databases its developers are using. The whole point is that web apps are supposed to be composed of services where the backend can be developed independent of the front-end. Nicholas Zakas talked about that very notion back in 2013.

On Meteor, perhaps Tom Dale said it best in this tweet:

 

And again in a recent blog post:

Unfortunately, when most people think of client-JavaScript-running-on-the-server, they think of technologies like Meteor, which do ask you to write both your API server (responsible for things like authentication and authorization) and your client app in the same codebase using the same framework. Personally, I think this approach is a complexity and productivity disaster, but that deserves its own blog post.

So, if not Meteor, then what or who? The company that can upend the world of enterprise application deployment and make it easy for everyone…well that’s one worth building, don’t you think? That’s the dream we’re chasing here at Aerobatic. More on that soon.

Zulily – Out of Site, Out of Mind?

A few days ago, Zulily released its Fourth Quarter and 2014 Full Year Financial Results.

  • Q4 net sales were up 52% year over year to $391.3 million
  • Q4 EBITDA $20.3 million
  • Active customers were up 54% to 4.9 million * Active is defined as bought >=1 in the trailing 12 months
  • Average Order Value of $58 (up 3% YoY)

Problem is, they missed expectations on both top line revenue and earnings per share. On the latter metric, they missed estimates by more than 40%. During the earnings call, they cited increased churn among more recent customer cohorts. That same day, they announced their CFO was departing amid admissions of wobbly execution.

“We have been growing so fast that, at times, we’ve let execution be more wobbly than it should be…” – Mark Vadon, Co-Founder Zulily

Gelatinous execution aside, all of this got me thinking. Arguably, the Zulily storefront is not actually Zulily.com. Instead, their storefront is actually a daily customer email. In that construct, you can think of someone’s Gmail inbox as being analogous to a mall or downtown core with the promotions tab the least trafficked bit of downtown. Not on the main street. Still downtown but a few blocks away such that serendipitous footfall is the exception rather than the rule.

As we’re seeing the great fragmentation of online retail play out with a proliferation of category challengers competing against traditional multi-category retailers and brands going direct to consumer, one effect is that we’re being saturated with more communication, email especially. One way to compete against the noise is to personalize the communication in the hope of being more relevant, but if Zulily is seeing more churn among customer cohorts, there’s just that much less data on a given customer with which to personalize and there’s only so much cute algorithms can accomplish.

In such a competitive environment, it’s little wonder we’re seeing more retailers and brands that got their start online, now find ways to have a presence in those physical downtown shopping centers either by opening their own stores, having pop-up stores like Warby Parker’s bus, or partnering with larger retailers a la Bonobos.

Downtown Seattle is and always will be composed of (not comprised of) not much more than four dense blocks of retail with a less trafficked outer perimeter. When people are out for lunch, walking to work, going to the movies, or grabbing a coffee, they simply cannot ignore the Sephora at Westlake, Banana Republic on 5th, Nike on Pike Street, etc. It’s stating the obvious, but those storefronts occupy space that no other retailer can occupy simultaneously. That’s not true of our email inbox, however. It can expand to accommodate more or less an infinite number of promotional emails and thus, for an online-only retailer to capture our attention on a consistent basis via their email storefront, it’s asking a lot.

Might we eventually see Zulily roll the dice and try to establish a physical presence, or is this just a temporary case of out of site, out of mind?

Old Habits Die Hard

Last Fall, I attended the AWS Re:Invent conference along with 13,000 other techies from around the world. During the event, Amazon went to great lengths to showcase the very large enterprises and government organizations that have adopted cloud computing.

Indeed, a good proportion of attendees were on the infrastructure teams of those large corporations, responsible, in part, for figuring out this cloud computing thing, and ultimately transitioning their company’s infrastructure onto AWS.

Undoubtedly these corporations will benefit in the long-run from elastic computing resources, but what could throttle those benefits are old processes that are being ported over to the new environment. For example, as an application developer in a large corporation, the typical process for starting a new project is to submit a ticket, wait for an infrastructure engineer / architect to be assigned to your project, discuss your needs, do some capacity planning, provision a virtual machine or buy some servers, maybe give you root access, but more likely, have you tell them what you need installed, they’ll take care of it for you, and eventually your environment is ready to go after a few back and forths. I’ve personally seen this end to end process take more than 100 days…

The cloud is supposed to end that inefficient process though, right? Well, not necessarily. While it’s true that the corporation will probably reduce their infrastructure costs by moving to the cloud, it’s not a guarantee that the newly empowered application developer will be able to move so much faster. I mean, there are only so many devs that know how to set up a cloud computing environment.

aws-console

For example, expecting a front-end developer whose skills lie with HTML, CSS, and JavaScript to now learn the myriad AWS tools to deploy their apps is simply not realistic. In fact, as Amazon continues to innovate, it’s likely the role of AWS architect will become ever more specialized – the AWS security architect, the AWS networking architect, the AWS analytics architect, the AWS database architect, the AWS storage architect, etc.

So now, we’re right back to where we started where an application developer needs an environment, but they don’t know the whole AWS stack, and they’re now working with their company’s AWS architect whose job now is to provision AWS resources instead of the old on-prem resources they used to do. Granted, ordering the server has been cut out of the equation but the flow could be better. Much better.

Over the next few years, we’re going to see more startups focus on specialized areas of the typical enterprise IT department to provide simplified, abstracted AWS-based services that will be composed of multiple AWS tools. Of course, AWS themselves are doing this to some extent but right now their major opportunities lie simply in getting these huge enterprises onto the cloud. As they further penetrate the mid-market, those abstracted services will become even more important where the “IT department” can sometimes be just a handful of folks.

In the meantime, corporate IT infrastructure teams would be wise to think not only about ditching their on-prem servers, but how the developers within their organization will actually provision and use those cloud services. At Aerobatic, the company I co-founded with this guy, we spend a lot of time thinking about how we can make the lives of front-end developers easy, but that’s only half the story. The other half is thinking about how to make the lives of the infrastructure architect easy too, such that, in the end, everyone’s happy, and those build-deploy-test cycles can keep pace with the business.

How Big Companies Lose

The optimistic title of this post would have been, “How Small Companies Win” but I’m Scottish, and so the pessimistic title it shall be.

To some, particularly those in tech, there’s a belief that the average big company “doesn’t get it.” The startup sees the world in a way that the large company does not. The startup sees a large market, a catatonic customer base ready to be awakened, and a host of large incumbents ready to be upended – The fat cats will get their comeuppance and won’t know what hit ’em.

However, as with most things, I’d argue it’s a bit more nuanced than that. Senior executives (at least the ones I’ve known) of large companies tend to be keenly aware of the macro conditions surrounding their business. They care deeply about their business. They obsess about the health of their business and they’re no less formidably educated, talented, or hungry than any startup counterpart. In fact, more often than not, their experience at large companies has afforded them a network that gives them access to the brightest minds and exposure to the latest ideas and trends in their industry. Given the available resources of most large companies, these executives have the latitude to chart a course that most startups can only fantasize about.

So, given these handicaps, why is it that we see large companies repeatedly stumble and startups make deep inroads into seemingly heretofore impenetrable markets?

I’d argue there are four primary reasons how big companies lose (and some secondary ones too):

  1. Focus – Large companies are often attacking and defending on multiple fronts. In any such approach, it’s inevitable that the best resources are diluted across those efforts. With startups, they tend to be (at least initially) singularly focused and that makes them dangerous.
  2. Risk – Startups have everything to gain. Publicly traded companies have everything to lose. Seemingly trivial issues can become major thorns and the tendency is toward risk avoidance, let alone risk mitigation, particularly when you traverse down from the execs and into the broader org at large.
  3. Heritage – Heritage needn’t be an abstract term to politely describe those long-since-forgotten values of a bygone era. Rather, heritage can be something tangible. Something you’re famous for. Something, when everything is going to hell in a handbasket, you don’t panic. You know what you’re about, and you’re not about to lurch. However, heritage can also mean being encumbered with the multiplicative choices of the big company’s forebears, and nobody ever bats 1000…
  4. Talent – Inevitably, within a large org, it’s impossible to stack it only with A-players. Or, said more accurately, I’ve yet to observe a large company with only A-players. Often with no major financial upside, a risk avoidance culture, and perhaps a difficult legacy to be inherited, up-and-comers understandably may be reticent to join the ranks of a big co, and ambitious and talented employees may feel their best chance of success lies outside the big co.

How big companies win is to address these issues head on. Acknowledging that every minute of the day, one or more startups are actively plotting to take their customers, to carve out a large piece of what was once “theirs” is a necessary starting point. That feeling has to be shared not just among the most senior executives but throughout the entire org. After all, the impermanence of success is the only permanent condition.

Time To Level Up

Earlier this year, I had the good fortune to meet Marc Strigel, COO of Soundcloud. Marc’s one of those rare folks that immediately strikes you as perpetually curious, always deconstructing and reframing what he thinks he knows, stress testing those constructs in conversations with others, giving you a lot to think about…During one of our conversations, he described the culture at Soundcloud, and one of those aspects stuck with me throughout the summer – the notion of leveling up.

From their site, here’s how Soundcloud describes it:

We use a term to describe how we’re constantly seeking to improve: “Level Up”. It’s about being self-motivated and challenging each other at every moment.

In America at least, we lionize people who push themselves to the absolute limit. Indeed, I used to be one of those people myself. In college, I ran division 1 track, and the mind games I played to convince myself that I was stronger than my opponents, willing to push myself past the point of exhaustion to prevail, were, in hindsight, the product of unadulterated obsession. Over time, this mentality comes to define who you perceive yourself to be and how you outwardly project yourself. The willingness to endure pain becomes a fact of life and as your ability progresses, so too does your pain threshold, and the addictive nature of it just takes over and the cycle repeats ad infinitum.

…But, that was 20 years ago. As I ran along the Ship Canal in Seattle last night, I ran with nothing but love and happiness. Did I push myself to the limit? Not even close. Indeed, I think now, I’m not looking to push any single part of my life to the obsessive limit. Instead, I’m trying to push all parts of my life to the point of happiness, or equilibrium, and that goes for my professional life too.

This week, I found happiness in figuring out how to write some site scraping code of all things. I found happiness in getting back the results of a formative product my team has been working on and seeing the enthusiasm it generated within the organization. I found happiness in a really interesting project I’ve been contributing to with a friend. I found happiness in writing.

At the time I spoke with Marc about Leveling Up, I mostly thought about the pressure for individuals of an organization to constantly push themselves to the limit and the potential for that to negatively manifest itself. Soundcloud is right though – the key really is self-motivation as opposed to external, organizational pressure or an internal fear of failure. The role of a leader in such an environment is not to create a culture of negative up-or-out energy, but to create the positive conditions that allow and encourage everyone to push themselves to the point of happiness. And real happiness comes not with rote repetition, stagnation, or fear, but with enlightened growth in all areas of your life that are important to you.

This weekend, I’ll be out there again teaching myself to fish for trout in some mountain lake, fumbling around with lures, casting bubbles, and leader lines – stuff I honestly had no clue about 3 weeks ago, but my kids will surely appreciate it, and that makes it important to me. Time to Level Up!

Jason-at-LSU

Business Intelligence Is Dead

The human brain constitutes about 2% of our body mass, but consumes as much as 20% of our energy. As an intermediary in decision-making, not only is the human brain inefficient, it’s also often wrong as discussed at length in Daniel Kahneman’s book Thinking Fast & Slow.

At the same time, computation is faster and cheaper than ever with no end to improvement on the immediate horizon. Combined with the rise of the Open Source movement, this has brought us to the point where machine learning and other artificial intelligence techniques are no longer the domain of obscure research papers but are now table stakes for most apps being built today.

In light of these advances and our inherent human fallibility, any business software that does not currently “close the loop” between analysis and action is essentially dead – a piece of zombie code used by people living in the past that rely on the output of these systems to prop up their vainglorious assertions with a convenient percentage here and a tortured average there.

The promise of Business Intelligence was always supposed to be about better decision-making, but more often than not, it provided nothing more than a prop for people to continue to decide emotionally and justify intellectually – a conversation piece, and an expensive one at that.

Consequently, the value of today’s Business Intelligence tools are essentially being discounted down to zero because of the humans that consume the output, belabor its meaning, and at some point long after the fact, make a decision that has a not insignificant probability of being wrong.

For enterprise SAAS companies that are being founded today, the basic act of reporting is a service that they will give away for free, because they understand that gathering & integrating data, labeling data, creating metadata, and performing transformations are only an intermediary step towards action within a closed loop system which is where the real value lies. Within such services, reporting of aggregate descriptive data is a convenient by-product and one that can be given away for free with relatively little incremental cost. Just as big-box retailers have their door-busting deals on soda, SAAS companies can afford to lead with free reporting knowing they’ll make it up and then some with much more valuable services (and ones that lend themselves better to pay-for-performance subscriptions).

If you are in the reporting business be it sales data, social data, web data, or any other variety of data, and you haven’t transitioned towards something further up the value chain, I would be very worried for what’s about to come. BI is already dead and it’s just a matter of time before the markets acknowledge that en masse and value it accordingly.

p.s. I spent a fantastic 4.5 years of my career  with MicroStrategy in the mid-2000s. During my time there, I would often hear Michael Saylor, the CEO of MicroStrategy make sweeping declarative statements about the future delivered with such conviction that they were taken as fact by most in attendance. With much affection, this post is partly for him. What’s up Mike?!

A Word On Segmentation

This was supposed to be a post on the rising cacophony around segmentation, but instead, the Google Trend of “segmentation” quickly disabused me of any such notion.

segmentation

For good measure, I also checked to see how “clustering” is trending, and it’s not any better…

clustering

Sadly, even a specific algorithm like k-means isn’t enjoying a halo effect from the sexiness surrounding all things data sciencey…

kmeans

It’s also of little comfort to see that k-means is 5X more popular among our Asian counterparts as compared to the U.S.

kmeansregion

So, what’s the point? Well, the point is that when you work in a company and lots of people across the spectrum of technical competence discuss the topic of segmentation, there exists an inevitable gulf between a primarily creative group of practitioners within the company who think of segments in terms of customer demographics and seek to create archetypes / personas to represent those customers, and those who work in a technical capacity and think primarily in terms of customer behavior.

Being able to bridge that gulf in thinking such that the creative parts of an organization can embrace a systematic, behaviorally-driven approach to segmentation has to be an important goal of any data scientist / statistician working on this problem. Sometimes, the ability to describe your work is more important than the work itself…