Never use Web UI or GUI to make changes to systems , but instead , use version control , git , and pull request , merge request model instead , adopt a complete software engineering workflow for system operations
If it doesn’t provide an API for management, configuration and other major tasks , Its Evil , don’t use it. Ever
Always work towards optimising the common case and don’t waste time on automating something that you need to do once a year.
Never make a lot of commits for a lot of changes , instead , group your changes into related change sets and them commit them , to keep your CI/CD pipelines clean as well as git history short.
Never push directly to git repository and try to make a merge request instead , it will give you and your team to have a second chance to review the changes
Never commit changes , even locally , before linting , building , testing it locally , to keep your commit history clean and neat , and avoid repeating a long pipeline
Never use a production system for testing something else , instead , you must be able to clone the production system into a testing version , do the testing there instead and trash everything after testing
Forget about the idea that you can scale operations without writing significant amount of code
The basic requirement for every system admin is to be a developer who has deep understanding of systems and loves to code them
It might be a bad idea to spam your users with announcing down time for every upgrade of the system , upgrades should be automated and safe. Facebook and Google never announced it from start to present while running systems that are making money. Instead , keep and define an Error budget , SLAs and SLOs for your system/service.
Always keep the backups , a failover system is not a backup though.
Don’t use RDMS for everything , maybe its not needed , maybe you can just use simple noSQL solution , or a key value store.
Don’t believe anything or anyone , until and unless you test it yourself , or re produce the actual behaviour of the system, don’t listen to doubts and assumptions.
Keep the configuration out of the software itself , for example , building the same docker image for all test , stage , prod environments , but the deployment manifest of that image is different across the environments , using the environment variables in your delivery system. Don’t bake the configuration into the image itself.
Always use multi-stage docker builds and never leave the dev tools in the production image , never , ever.
Don’t believe in workarounds , address the original issue
To be successful , embrace the change at constant rate