These books cover design trade-offs to consider when scaling systems, the cost and benefits of each design choice, and how current technologies embrace and influence these design choices. They do so while being focused on practice.

Books 1 and 2 are more high-level and cover aspects relevant when designing and operating the system. They are light on nitty-gritty low-level design/implementation details. …

Recently, while writing an article about logging in Python, I started identifying artifacts and actions that we commonly use when we talk about logging. Here’s what I came up with.

When we use a logging library, we perform/trigger the following common tasks while using the associated concepts (highlighted in bold).

1. A client issues a log request by executing a logging statement. Often, such logging statements invoke a function/method in the logging (library) API by providing the log data and the logging level as arguments. The logging level specifies the importance of the log request. Log data is often a log message, which is a string, along with some extra data to be logged. …

Thoughts on why+how container technology influenced software development

Why?

1. Containers serve as commonly agreed upon deployment units between Dev teams and Ops teams, and this eliminates the possibility of missing artifacts when a Dev team hands over a program to the Ops team for deployment.
2. Containers allow Dev teams to package any software required by the deployed program, and this eliminates the need to provide a list of required software to the Ops team when providing the program for deployment.
3. Containers allow Dev teams to test the deployment of programs independent of the Ops team and the actual deployment environment.
4. Containers allow Dev teams to test the program as a deployed unit independent of the Ops team and the actual deployment environment. …

Few things to ponder about; more so, if you deal with data.

1. If it has become cheaper to produced a new invention, this suggests that we are using our information wisely and are forging it into knowledge. If it is becoming more expensive, this suggests that we are seeing signals in the noise and wasting our time on false leads.
2. Data-driven predictions can succeed — and they can fail. It is when we deny our role in the process that the odds of failure rise. Before we demand more of our data, we need to demand more of ourselves.
3. In the paper, he [Akerlof] demonstrated that in a market plagues by asymmetries of information , that quality of goods will decrease and the market will come to be dominated by crooked sellers and gullible or desperate buyers. …

Recently, while reading about MPI, I stumbled on the mpi4py library that brings MPI to Python. Given my past familiarity with Python’s multiprocessing module, I was curious how would the performance of MPI compare the performance of multiprocessing module. So, I performed a small experiment.

Computing Julia Set

Since mpi4py is based on MPI, it exposes many MPI-based features that are not readily offered by multiprocessing. Also, via mpi4py.futures, mpi4py offers features such as map and starmap that are similar to those provided by multiprocessing. So, I decided to compare the performance of these modules based on the map function.

The first example in mpi4py documentation was an ideal candidate for this purpose for two reasons. First, the example computes the Julia set of dimension 640x480 pixels and dumps it into a PGM file, a highly parallel computation. Second, the code was structured such that replacing mpi4py with multiprocessing required changing only three lines!! …

Truly hiding implementation details

In object-oriented languages like Java, C#, or Kotlin, given a type T, an associated view type TView is used to expose a specific view (parts) of an object of type T. This helps hide implementation details.

For example, in the following Kotlin example, Ledger interface is used to provide access to a ledger while hiding the underlying implementation details, i.e., LedgerImpl provides the functionalities of a ledger and it has a process and container members.

interface Ledger {
    fun getValue(i: Int): Int?
}class LedgerImpl: Ledger {
    val container = HashMap<Int, Int>()
    override fun getValue(i: Int) = container.get(i) …

Thundering Web Requests: Part 6

This is the sixth and final post in a series of posts exploring web services related technologies. It documents the observations about custom web clients that were implemented in Elixir, Go, and Kotlin and used to evaluate web service technologies.

Observations based on Minimum Time per Requests

While analyzing the behavior of various web service implementations, I observed that minimum time per requests for different web clients were starkly different. So, I plotted their minimum time per request against Actix-Rust service implementation as it was one of the most performant and reliable service implementation in the previous experiment.

On the topic of “excellence as a core strategy”, the book does a great job of providing great insights and quite a bit of food for thought and practice.

As for presentation style, it uses a style almost identical to that of “The Little Big Things” by Tom Peters. While I would have preferred a bit more structure, this style made is an quick read. So, no major complains here :)

As for highlights, the layout of the book makes many text fragments highlights — literally :) Even so, I figured I’d capture some here; a few of these are quotes. …