Oct 28, 2016

Simplifying Distributed Systems Using Apache Kafka

With the rising popularity of micro service architectures and typical scaling patterns used at enterprises, distributed systems are becoming more common and complex. What was once a simple web server connected to a database now can entail multiple databases, caches and integrations to other services/systems. By using Apache Kafka one can take much of the integration complexity out of the system, reduce coupling between the different components, easily expand functionality without disruption and scale horizontally.

This presentation will cover patterns and concepts that can be used to achieve all of the above. There will be a quick overview of how Apache Kafka works, it’s differences from other messaging brokers and why that’s important. I’ll speak about the good, the bad and what was missed during my experience working with large distributed systems. Finally I’ll briefly mention other technologies that work well with Kafka.

Original slides here.

About the Author

Object Partners profile.
Leave a Reply

Your email address will not be published.

Related Blog Posts
Android Development for iOS Developers
Android development has greatly improved since the early days. Maybe you tried it out when Android development was done in Eclipse, emulators were slow and buggy, and Java was the required language. Things have changed […]
Add a custom object to your Liquibase diff
Adding a custom object to your liquibase diff is a pretty simple two step process. Create an implementation of DatabaseObject Create an implementation of SnapshotGenerator In my case I wanted to add tracking of Stored […]
Keeping Secrets Out of Terraform State
There are many instances where you will want to create resources via Terraform with secrets that you just don’t want anyone to see. These could be IAM credentials, certificates, RDS DB credentials, etc. One problem […]
Validating Terraform Plans using Open Policy Agent
When developing infrastructure as code using terraform, it can be difficult to test and validate changes without executing the code against a real environment. The feedback loop between writing a line of code and understanding […]