We have an array of wonderful tools and platforms available to help us automatically monitor, scale, heal and failover our servers. Designing our products for Five Nines of availability (99.999% or just over 5 minutes of downtime per year) is absolutely achievable in 2017. However, this may not be necessary for your business. Here’s why:
- High availability is expensive. Maintaining multiple servers across multiple data centers incurs time and money, and it’s difficult to justify this “just-in-case” business expense.
- Redundancy requires complexity. If your team doesn’t have the necessary expertise to manage this complexity, continuing to add Kubernetes clusters and redundancy could even make things worse.
- If you’re relying on trusted third party providers, it often doesn’t make sense to build in redundancy. For example, when Amazon S3 suffered a 5 hour breakdown in early 2017, very few companies had an automatic failover built into their architecture. This is because the price of maintaining it vastly outweighed the likelihood of S3 going down. While you could back up your third party DNS, CDN and cloud server… is it really worth it?
For most of us, downtime is inevitable. While you can’t control everything, you can manage the experience customers have when your services do start to go down.
In this article, we look at how teams can design their downtime to be less disruptive to customers, and reduce the cost of this downtime to the business.
Prioritize Your Effort
Depending on your business type, different parts of your service will be more critical to maintain than others. What are the most important actions customers need to perform? These are the ones that must be protected first as a priority.
Prioritize redundancies by deciding how critical each service is to the running of your product. Is your ecommerce site primarily a catalogue? If so, your slave SQL database won’t need to be promoted to master when things go wrong, as users will still be able to browse your products. Is search critical to your site? If not, you can take the money set aside for your redundant ElasticSearch instance and invest it elsewhere.
With unlimited time and resources, we could have a bulletproof service that never goes down. But that’s not the case for most of us. (If this is you, and you’re hiring, please give me a call…). Instead, we need to prioritize our time to make sure the most critical services stay up.
If one tiny error brings your website crashing to its knees and flashing 500 status code errors, you’re going to upset a lot of customers. Your service needs to handle brief moments of instability or unavailability gracefully. For example, Google offers an HTML only version of Gmail when loading is slow. This means that customers can still get the information they require, albeit without the full experience.
Offline First is a popular trend at the moment, especially for mobile. It involves caching data and assets on the client so that the service still works when the server is unavailable. Its primary focus is for mobile – 4G is widely available, but deteriorates when customers are on the move, such as on a train. But even if your app is designed for desktop browsing where the connection is generally stable, Offline First will allow your app to continue functioning if your server goes down. Obviously a customer would have to visit your site or download your app beforehand, so this technique will only help for returning customers, and not new ones.
For the web, the service worker is a great way to start implementing offline-first paradigms. It’s a programmable network proxy that lives in the browser which allows you to intercept network requests as they happen. The service worker also has access to the browser cache, meaning it can store the results of network requests for future use. If a customer makes an HTTP request, and the server returns a 500 error, the service worker is able to intercept the error and instead return the last version of the request that was successful.
Single page applications – web pages that load a single HTML page and update dynamically as the user interacts with the app – can use other techniques for gracefully degrading. Since network requests are primarily made using Ajax, you can control the user experience after receiving a 500 message. For example, if your search server is down, an inline message can be displayed with more helpful information, rather than just serving up a generic error page. If you’re storing data in the web browser’s local storage, you could even offload search to the client to provide a degraded version to the end user. Local storage can also be utilized to store user provided data, allowing a form to be resubmitted automatically when your server is back online. Clever use of all of these technologies, along with good UX to set customer expectations, means you can still provide a stable service when your servers are burning!
Understand How Your Services Depend On Each Other
It’s important to keep track of your architecture, how different services depend on each other and what happens to the end user if one of these services is unavailable. For example, a common SaaS stack would include a key-value store like Redis. How does your application react if Redis runs out of storage, is overloaded or goes down? If you’re only caching data in Redis, it should be possible for your service to continue running when it becomes unavailable. If you’re storing session data in Redis, what happens if a customer tries to log in? Can you fall back to a different mechanism, like file storage? Does the affected customer see a sensible error message or just a 500 page? Keep these eventualities and possibilities in mind when adding services to your stack, and you’ll be able to keep your customers satisfied when the worst happens.
Keeping Customers Informed
While you’re working on bringing service back up to normal, your customers are waiting on you. They don’t know what’s happening behind the curtain, and they don’t know if the downtime will last a few seconds or many hours longer.
The only way they can know what to expect is by your communications with them. 88% of customers have lost faith in a company that experiences frequent downtime. But transparency during the downtime can go a long way to repairing that trust. 94% of customers are likely to be more loyal to a brand that commits to full transparency – even when it’s not pretty.
Keep in mind:
- Don’t make promises you can’t keep. Exact time estimates are the worst sins. If you suggest the service will be back up in 10 minutes, and it actually takes 15, you’ll have a lot of frustrated customers knocking on your door!
- Don’t throw your vendors under the bus. Even if it is the fault of your DNS, you chose them in the first place, and you didn’t back it up!
- Don’t “apologize for the inconvenience”. It doesn’t mean anything to customers any more. Instead, offer a genuine, specific apology for the trouble they are seeing.
Status pages that function independently of your architecture, like Statuspage.io or Sorry App, are a good investment. They’ll provide an easy way for customers to keep apprised of the situation, as long as you keep them updated. In-app notifications can also help explain why customers are seeing unusual behavior, but are more difficult to set up on the fly. Working with your marketing team to update social media feeds or send emails to key customers might be an easier way to spread the word about degraded performance.
When service resumes, it’s often helpful to provide customers with a post mortem. This can include a brief explanation of what went wrong, how it was fixed and what you’re doing to ensure it doesn’t happen in the future. Again, customers trust transparent companies, so being open post outage can win you back some of the trust you lost.
Working With Your Tech Support Team
Most of the time, your engineering team won’t be dealing directly with customers. Instead, you’ll have an army of customer support agents between you and the angry mobs trying to access the site. These people are your friends – they transfer information from your customers to you, and vice versa. Relying on your customer support team to communicate with customers means you can focus on getting everything back online.
To work effectively with your frontline teams, you must have processes set up before the storm hits. Know who your Specific Point of Contact (SPOC) is and how best to contact them. Understand what information they need to provide good customer service to customers throughout the outage. Let them know what information from customers is helpful. Do you want error codes? Troubleshooting reproductions? Your customer support team can get all of these for you.
Working with your tech support team will help ease the effect on customers, and hopefully help you identify the root cause even faster.
Planning For The Worst
Balancing the need for high availability with available resources inevitably results in some tradeoffs. But the good news is that you can always help direct how your customers experience service degradation.
With proper planning and communication, your customers will stick around, even when you go down.
It’s no exaggeration to say that big data is big business nowadays. Research by IDC suggests the big data analytics market is growing at 11 per cent each year, and will soon be worth more than $200 billion. That’s not entirely surprising when you consider the predictions for future data volumes, with the ten zettabytes of data generated in 2015 expected to increase 18-fold by 2025.
Bitcoin came onto the scene in 2009, and although its source remains anonymous it has become a major form of currency in the global market. Bitcoin’s validity was debatable at first, but skeptics were quieted when its value rose quickly; it has fluctuated greatly since then, yet it is still becoming an increasingly popular payment method for e-commerce businesses and online sales.
Is accepting bitcoin right for your business? The answer for most businesses will be yes, however there are important considerations to make when thinking about accepting bitcoin for online purchases. Let’s hold the digital magnifying glass up to bitcoin and see if it’s right for you.
First things first: what is bitcoin?
Bitcoin is a digital decentralized peer-to-peer cryptocurrency.
Wow – now that’s a very cryptic description. Let’s break it down. When we say bitcoin is decentralized, that means it doesn’t belong to a particular country or bank, and thus is not subject to a single administrator or governing body. For example, there is the U.S. dollar, which belongs to the United States and falls under the jurisdiction of the Federal Reserve; there is the pound sterling, which belongs to the United Kingdom and is controlled by the United Kingdom government; then there’s bitcoin, which belongs to no particular country and follows no set rules.
Bitcoin was created as a decentralized currency to allow users to have greater freedom with the way they stored and spent their money, and also to prevent a person or group of people from enforcing nefarious control over its value. Erik Voorhees writes for Bitcoin Magazine, “Consider: since every CoinBase [one of the major bitcoin transacting platforms] user can opt out and leave the platform, this presents a natural check on CoinBase’s ability to act with impropriety, and makes coercion impossible.”
Next in that bitcoin definition: bitcoin is fully digital. You will not have a smattering of bitcoins at the bottom of your sock drawer after an international vacation. There are no printed notes or minted coins; bitcoin exists solely as numbers on a screen. All transactions are completed online, following the increasing trend of the digitization of companies by fintech companies and major banks alike.
Bitcoin transactions are peer-to-peer, meaning they are sent directly from one individual to another. Your transaction doesn’t have to go through a bank, but is rather sent from one person or organization to another using a unique, encrypted code. This means that fees are very low, with some companies not charging any fees at all. An individual holds their bitcoins in what’s known as a wallet, and there are multiple companies that offer these wallets to consumers and businesses.
Finally, regarding bitcoin being a cryptocurrency, this has to do with the way transactions are processed. The codes that are used to facilitate the exchange of bitcoins employ cryptography to ensure their safety. The individuals’ identities are not revealed, but a public record of the transaction is available, making this a highly secure yet fully transparent monetary system.
The big question: should I accept bitcoin at my e-commerce site?
There are both pros and cons to accepting bitcoins, but currently the good outweighs the bad.
Your first consideration should be the size and number of transactions your business tends to make. If you are a large company and your transactions are few but large, bitcoin may not be advantageous in the current currency environment (although, of course, that may change). However, if you’re a small company or your transactions are frequent and in small amounts, bitcoin could offer huge savings in transaction fees.
Credit card companies charge fees to businesses that tend to be between 2% and 3%. As mentioned previously, bitcoin’s transaction fees are between 0% and 1%, netting a potential total savings of 1% to 3%. If you’re a small business owner, you know that 1% to 3% can make a huge impact on your bottom line, thus the savings from accepting bitcoin on your e-commerce site could be substantial. Since small businesses do not operate on the same scale as larger companies, they do not have the bargaining power to negotiate lower rates with credit card companies. Therefore, they stand to save a massive amount by accepting bitcoin.
Another benefit to using bitcoin is that bitcoin transactions complete very quickly, in a matter of minutes, as opposed to days as with credit cards. Getting paid quickly is very important for small and medium businesses who require a steady cash flow to maintain operations. The nuts and bolts of how transactions are processed is where things can be somewhat difficult to understand, but we’ll give it a try: after a bitcoin transaction is initiated, it needs to be verified by a bitcoin miner – that’s someone whose job it is to process bitcoin transactions. Since this is an individual and not a company, financial institution, or governing body, it keeps with the decentralized model of the bitcoin currency. Once a miner verifies, or “solves,” a bitcoin transaction the funds are then transferred from one account to the other.
Although this process may sound time consuming, it usually takes about 10-20 minutes from start to finish, or up to an hour if network connectivity is below par. This is exponentially less time than credit card transactions or wire transfers, which take days to complete, and this time saving can prove hugely beneficial to small and medium businesses. After all, time is money.
The next benefit to bitcoin transactions is that they are final, which means that the chargebacks or returns that sometimes occur with credit card transactions cannot and do not happen with bitcoin. Customer disputes can delay credit card payments and disrupt cash flow, and can also be quite costly, charging the merchant a fee between $5 and $20 per chargeback. If chargebacks occur too frequently, a business can be placed in a chargeback monitoring program, the fee for which is a costly $5,000. Bitcoin’s rapid transaction speed negates the possibility of chargebacks entirely: when a customer makes a purchase, it happens in a matter of minutes and with no room for second thought. This does not, however, mean businesses can be foolhardy with their practices, as online reviews are still alive and well, so business ethics are not to be sacrificed.
Businesses that have international customers also stand to benefit by using bitcoin; in addition, companies that have been unable to go global due to size restrictions or foreign transaction charges will find accepting bitcoin an easier way into the international market. No matter the financial institution or country, banks that facilitate purchases across borders will find a way to get their cut of the deal in one way or another – after all, those fees should be going to the business and not the bank. On the other hand, because bitcoin is decentralized, businesses will never have to pay foreign transaction fees.Businesses will, however, need to consider fluctuations in currency rates.
Finally, businesses that accept bitcoin can enjoy an improved public perception by being on the cutting edge of technology. Bitcoin is the new kid on the block when it comes to currency, and carries with it the appeal of being the coolest and most hip type of currency today. By accepting bitcoin, businesses piggyback on bitcoin’s state-of-the-art feel and will be thought of by their customers as a pioneer in e-commerce.
What are the risks associated with accepting bitcoin?
As with any online exchange of funds, there are risks to take into account when considering bitcoin as an online payment method. While bitcoin’s being decentralized is one of the key benefits bitcoin users will proselytize, on the flip side that may scare some merchants. Even though almost all major countries accept bitcoin as payment, some do not, reflecting market uncertainty. Some argue that bitcoin being decentralized means it is more likely to fluctuate, while others insist that its decentralization is precisely the reason it will not fluctuate, since it does not depend on a single country or institution to determine its stability.
Bitcoin has seen massive fluctuation since it was created, which also means it can be difficult to price goods accurately without knowing the current exchange rate of bitcoin with local currency. This could be a significant challenge for an accounting department, as businesses may need to be able to change the price of their goods or services almost daily to correspond with bitcoin fluctuation.
How can I get started with using bitcoin?
If a business would like to start accepting bitcoin payments but expects small numbers of transactions, it might want to begin by simply by creating a bitcoin wallet and advertising on the website that it accepts bitcoin, and to contact the business directly to facilitate payments on a case-by-case basis. This is an easy entry into the bitcoin platform.
For businesses that want to really get the ball rolling, they will want to use a bitcoin payment company. There is a handful of these companies on the market today, and each one offers a slightly different solution to accept bitcoins. Check out Bitpay, Coinbase, and Stripe to see which one works best for your online business.
Despite its anonymous cloak-and-dagger beginnings, in just eight years bitcoin has become a legitimate, internationally recognized global currency, not only with huge investment potential but also with multiple potential benefits to e-commerce businesses. Bitcoin’s positive attributes are many, although they must be tempered with the risks of a currency that has fluctuated a lot in its short life. Accepting bitcoins as a payment method is relatively straightforward, and can have great benefits for a company’s market perception. If bitcoin is the future of online payment systems, consider this article your crystal ball: we see bitcoin in your future.