Things to Learn and Un-Learn

  • Never use Web UI or GUI to make changes to systems , but instead , use version control , git , and pull request , merge request model instead , adopt a complete software engineering workflow for system operations
  • If it doesn’t provide an API for management, configuration and other major tasks , Its Evil , don’t use it. Ever
  • Always work towards optimising the common case and don’t waste time on automating something that you need to do once a year.
  • Never make a lot of commits for a lot of changes , instead , group your changes into related change sets and them commit them , to keep your CI/CD pipelines clean as well as git history short.
  • Never push directly to git repository and try to make a merge request instead , it will give you and your team to have a second chance to review the changes
  • Never commit changes , even locally , before linting , building , testing it locally , to keep your commit history clean and neat , and avoid repeating a long pipeline
  • Never use a production system for testing something else , instead , you must be able to clone the production system into a testing version , do the testing there instead and trash everything after testing
  • Forget about the idea that you can scale operations without writing significant amount of code
  • The basic requirement for every system admin is to be a developer who has deep understanding of systems and loves to code them
  • It might be a bad idea to spam your users with announcing down time for every upgrade of the system , upgrades should be automated and safe. Facebook and Google never announced it from start to present while running systems that are making money. Instead , keep and define an Error budget , SLAs and SLOs for your system/service.
  • Always keep the backups , a failover system is not a backup though.
  • Don’t use RDMS for everything , maybe its not needed , maybe you can just use simple noSQL solution , or a key value store.
  • Don’t believe anything or anyone , until and unless you test it yourself , or re produce the actual behaviour of the system, don’t listen to doubts and assumptions.
  • Keep the configuration out of the software itself , for example , building the same docker image for all test , stage , prod environments , but the deployment manifest of that image is different across the environments , using the environment variables in your delivery system. Don’t bake the configuration into the image itself.
  • Always use multi-stage docker builds and never leave the dev tools in the production image , never , ever.
  • Don’t believe in workarounds , address the original issue
  • To be successful , embrace the change at constant rate

Leave a comment

Your email address will not be published. Required fields are marked *