Rolling out is a topic I have always wanted to talk about. Especially focusing on when you roll out the first company project. Although there is no difference whether you’re startup or not. When it comes to rolling out a project you need to plan it carefully and make sure you keep the pace with your time to market deadline.
Timeline should be the first thing you should keep on mind towards the rollout. It will give you an indication what should be done and how effectively you going to use your resources to get it on time.
There is no second chance for being late to the rollout deadline. If you’re not focused it won’t get close to the finish line. When your business depends on that (e.g your first customer), it could be deadly for your company’s future.
Finalize your rollout features
Before rolling out, make sure all features which have been previously agreed upon about the product going are settled. Make sure all stakeholders know the finalized version content before the rollout. That will enable you to estimate your rollout plan. This isn’t the time to start changing things, unless they are real rollout breakers.
Is it necessary for the rollout?
Being a perfectionist before your rollout can play against you. You should pay attention to avoid those thoughts right before going live: “We can optimize this logic,” “We can squeeze that feature,” “Lets switch to that technology”, etc.
You must avoid this by all means. Always ask yourself, “Is what am I doing necessary for rollout?” New technology, or any new feature that isn’t necessary should backlogged but for sure shouldn’t be included in the rollout tasks. You must keep things simple and targeted. During the rollout period every discussion with my team regarding a feature or new implementation focuses on asking this question over and over again: “Is it necessary for rollout?” the answer always puts us back on track. If it is not necessary -> backlog !
Simulate day one
It’s important to understand that despite all your planning, after going live you’ll be exposed to new surprises. Mocking real user’s behavior is almost impossible. Things that we haven’t thought about or didn’t expect have the tendency to arise on production. Therefore, you must simulate real usage with your target audience right before going live. Force your team, management, and anyone else in the company to use the App/Service as much as possible before you go live. You’ll be surprised by the new things you’ll find. Things like no-connectivity, half-baked requests, weird UI behaviour and so on. Make sure you are semi-live some time before the first users enter to catch issues as early as possible.
Production support plan
So you are new startup or new big project within a stable company. Your baby is coming alive and you need to support it. If something goes wrong(e.g production incidents) who is taking responsibility? How will you support real-time issues? If your project is already based on big company this will probably be easier, as big companies already have their NOC team or any other production support tiers.
So what happens with new startups??
As always when you have to do something new and you are not sure how to tackle it I pick the “lean” way. Start small and improve on time and by demand.
Let’s assume your team is built by 5-8 people, you’ll need to take shifts. (I am already assuming that you have some monitoring and alerting system – if not, that’s out of scope of this post). Weekly shifts could be a good start. Each person has their turn to be the one who will accept incident issues after working-hours. The best way to manage this is to use 3rd party services like PagerDuty, OpsGenie, etc. You have no time to build this type of management system – better pay a few bucks instead. Make sure you fine tuned your incident alerts to be effective and minimize false alarms.
Incident day simulation
What I like to do before rollout is to plan an “incident day” with the team. On that day we mess with the system in order to practice real incidents situations. Each team member is selected randomly and tries to fix the issue. For example, we take the DB down which would raise up incident issues. the team members will practice how to investigate and react to these issues. That will help them to experience on ‘dry-mode’ in preparation for real time incidents.
Production is a holy place
I am sure that before your rollout you will use production for other matters (Testings, Demos, etc.) Not any more! Production will be dedicated to real users only. No more testings or experiments on it. If you have customers testing your system move them into other env(Demo env). Make sure you clean-up everything including old users and start fresh. Additionally, make sure production has its own infrastructure and own resources. Don’t share it with other env’s it will bite you. For example, if you using ELK stack, create separate instances just for prod. Last thing you need is to have log issues because someone on other env overloaded your logging system. I would do the same for any other infra matters(Queues, storage, etc..)
Rolling out is stressful stage but it can also be a lot of fun, so remember to enjoy 🙂