Designing Data-Intensive Applications

Title: Designing Data-Intensive Applications
Author: Martin Kleppmann
Publication date: March 2017
Print length: 616 pages

Chapter 1

Thinking About Data Systems

  • There are datastores that are also used as message queues (Redis), and there are message queues with database-like durability guarantees (Apache Kafka). The boundaries between the databases, queues, caches, etc. are becoming blurred
  • Increasingly many applications now have such demanding requirements that a single tool can no longer meet all of its data processing and storage needs. Instead, the work is broken down into tasks that can be performed efficiently on a single tool, and those different tools are stitched together using application code.
  • If you are designing a data system or service, a lot of tricky questions arise. How do you ensure that the data remains correct and complete, even when things go wrong internally? How do you provide consistently good performance to clients, even when parts of your system are degraded? How do you scale to handle an increase in load? What does a good API for the service look like?
  • Reliability roughly means "continuing to work correctly, even when things go wrong."
    • systems that anticipate faults and can cope with them are called fault-tolerant or resilient.
      • fault is not the same as a failure.  A fault is usually defined as one component of the system deviating from its spec, whereas a failure is when the system as a whole stops providing the required service to the user
      • it can make sense to increase the rate of faults by triggering them deliberately. by deliberately inducing faults, you ensure that the fault-tolerance machinery is continually exercised and tested
  • Scalability is the term we use to describe a system’s ability to cope with increased load
  • Defining load on your system by determining an appropriate load parameter for your data systems is key
  • When measuring performance, averages are not a good metric. It is better to use percentiles of 95 or higher. This will allow you to determine the outliers in your measurement. 


Now that we have discussed the parameters for describing load and metrics 

https://learning-oreilly-com.dclibrary.idm.oclc.org/library/view/designing-data-intensive-applications/9781491903063/ch01.html

Comments

Post a Comment

Popular posts from this blog

Azure Architecture Explained

Chapter 1 As organizations continue to decentralize and outsource non-core functions to suppliers, it’s essential to establish trust between partners and have transparency in data management to ensure data security and privacy IAM is a technology framework that helps organizations ensure that the right people have access to the right resources. IAM includes two main functions: authentication and authorization. IAM capabilities have become the central pillar for cloud-based scenarios A crucial piece of an IAM solution is its life cycle management capabilities. Microsoft Cloud Adoption Framework (CAF) provides a set of best practices, guidance, and tools for different stages of cloud adoption, from initial planning to implementation and optimization it is not just for Azure, it can work with  Amazon Web Services (AWS) and Google Cloud. includes a specific IAM design area that focuses on providing guidance and best practices for designing secure and scalable IAM solutions Azure AD sup...
Read more

The Ethiopian Army: From Victory to Collapse, 1977-1991

Title: The Ethiopian Army: From Victory to Collapse, 1977-1991 Author: Fantahun Ayele Publication date: 2014 Print length: 328 pages Introduction Comparison of Ethiopian revolution with French and Russian (3) Chapter 1 Under Tewodros, Ethiopia started producing mortars and cannons (5) How the Ethiopian army was modernized under Menelik and Haile Selassie (5-7) The British trained the Ethiopian army after the Italian liberation (7-8, 15-6) Ethiopia pulls away from England and becomes close the US (10) Ethiopia reorganizes military fearing a future Somalian attack (11) The imperial bodyguard attempts a coup (12) The imperial bodyguard always received special treatment. Troops that were sent to Korea were selected from the imperial bodyguard (12) Other forces of the military were jealous of the imperial bodyguard because of the special treatment they received (12) "the conflict between the bodyguard and other units did not annoy the emperor. After all, it was the emperor who presumab...
Read more

Building Microservices, 2nd Edition

Title: Building Microservices, 2nd Edition Author: Sam Newman Publication date: August 2021 Print length: 612 pages Chapter 1 Microservices are a type of service-oriented architecture.  Deployability is key for a microservices. to ensure independent deployability, we need to make sure our microservices are loosely coupled Having clear, stable service boundaries that don’t change when the internal implementation changes results in systems that have looser coupling and stronger cohesion. Model services around business domains it will make it easier to roll out new functionality and to recombine microservices in different ways to deliver new functionality microservices should be slices of business functionality that encapsulate user interface (UI), business logic, and data Microservices should avoid the use of shared databases if a microservice wants to access data held by another microservice, it should go and ask that second microservice for the data Microservice should be kept...
Read more