A few weeks ago, production environment came to an outage, solve it cost me 8 hours (from 3am to 11am) although total down time is not long, really a bad expenrience. Finally, impact was mitigated, and I’m working on a long term solution. I learned some important things from this accident.
The …
AWS’s DMS (Data migration service) can be used to do incremental ETL between databases. I use it to load data from RDS (MySQL) to Redshift.
It works, but have some concerns. Take some notes when doing this project.
Prerequisites Source RDS must:
Enable automatic backups Increase binlog remain …
I use autoscaling group to manage stateless servers. Servers go up and down every day.
Once server is up, I will add a PTR record for it’s internal ip. But when it’s down, I didn’t cleanup the PTR record. As times fly, a lot of invalid PTR records left in Route53.
To cleanup those PTR records …
Build static website on S3 is very easy, but by default, it can be accessed by open internet.It will be super helpful if we can build website only available in VPC. Then we can use it to host internal deb repo, doc site…
Steps are very easy, you only need VPC endpoints and S3 bucket policy.
AWS api …
Coredump file is useful for debuging program crash. This post will show several settings related to coredump.
Enable coredump If you run program from shell , enable coredump via unlimit -c unlimited， then check unlimit -a | grep core, if it shows unlimited, coredump is enabled for your current …
I’m lazy, I don’t want to be deb/rpm expert, I don’t want to maintain repo server. I want as less maintenance effort as possible. 🙂
Combine tools fpm, aptly with aws s3, we can do it.
Use fpm to convert python package to deb fpm can transform python/gem/npm/dir/… to deb/rpm/solaris/… packages …