Recently, I upgraded to R 4.0.1. To avoid broken packages, I started updating installed packages using installed.packages() and install.packages(). This solution was less than ideal because, if a dependee package was not built for R 4.0.1, then the dependent package failed to build and install.

After some search, I stumbled on update.packages(pkgs, checkBuilt=T, ask=F) solution. While the documentation of update.packages says, “If checkBuilt is TRUE, a package built under an earlier major.minor version of R (e.g., 3.4) is considered to be ‘old’”, this command did not update the installed packages that were built for R 3.6.2 (according to the output of packageDescription function). …


Image for post
Image for post

These books cover design trade-offs to consider when scaling systems, the cost and benefits of each design choice, and how current technologies embrace and influence these design choices. They do so while being focused on practice.

Books 1 and 2 are more high-level and cover aspects relevant when designing and operating the system. They are light on nitty-gritty low-level design/implementation details. …


Recently, while writing an article about logging in Python, I started identifying artifacts and actions that we commonly use when we talk about logging. Here’s what I came up with.

When we use a logging library, we perform/trigger the following common tasks while using the associated concepts (highlighted in bold).

1. A client issues a log request by executing a logging statement. Often, such logging statements invoke a function/method in the logging (library) API by providing the log data and the logging level as arguments. The logging level specifies the importance of the log request. Log data is often a log message, which is a string, along with some extra data to be logged. …


Thoughts on why+how container technology influenced software development

Image for post
Image for post

Why?

  1. Containers serve as commonly agreed upon deployment units between Dev teams and Ops teams, and this eliminates the possibility of missing artifacts when a Dev team hands over a program to the Ops team for deployment.
  2. Containers allow Dev teams to package any software required by the deployed program, and this eliminates the need to provide a list of required software to the Ops team when providing the program for deployment.
  3. Containers allow Dev teams to test the deployment of programs independent of the Ops team and the actual deployment environment.
  4. Containers allow Dev teams to test the program as a deployed unit independent of the Ops team and the actual deployment environment. …


Few things to ponder about; more so, if you deal with data.

Image for post
Image for post
  1. If it has become cheaper to produced a new invention, this suggests that we are using our information wisely and are forging it into knowledge. If it is becoming more expensive, this suggests that we are seeing signals in the noise and wasting our time on false leads.
  2. Data-driven predictions can succeed — and they can fail. It is when we deny our role in the process that the odds of failure rise. Before we demand more of our data, we need to demand more of ourselves.
  3. In the paper, he [Akerlof] demonstrated that in a market plagues by asymmetries of information , that quality of goods will decrease and the market will come to be dominated by crooked sellers and gullible or desperate buyers. …

Recently, while reading about MPI, I stumbled on the mpi4py library that brings MPI to Python. Given my past familiarity with Python’s multiprocessing module, I was curious how would the performance of MPI compare the performance of multiprocessing module. So, I performed a small experiment.

Image for post
Image for post

Computing Julia Set

Since mpi4py is based on MPI, it exposes many MPI-based features that are not readily offered by multiprocessing. Also, via mpi4py.futures, mpi4py offers features such as map and starmap that are similar to those provided by multiprocessing. So, I decided to compare the performance of these modules based on the map function.

The first example in mpi4py documentation was an ideal candidate for this purpose for two reasons. First, the example computes the Julia set of dimension 640x480 pixels and dumps it into a PGM file, a highly parallel computation. Second, the code was structured such that replacing mpi4py with multiprocessing required changing only three lines!!


Truly hiding implementation details

Image for post
Image for post

In object-oriented languages like Java, C#, or Kotlin, given a type T, an associated view type TView is used to expose a specific view (parts) of an object of type T. This helps hide implementation details.

For example, in the following Kotlin example, Ledger interface is used to provide access to a ledger while hiding the underlying implementation details, i.e., LedgerImpl provides the functionalities of a ledger and it has a process and container members.

interface Ledger {
fun getValue(i: Int): Int?
}
class LedgerImpl: Ledger {
val container = HashMap<Int, Int>()
override fun getValue(i: Int) = container.get(i) …


Thundering Web Requests: Part 6

This is the sixth and final post in a series of posts exploring web services related technologies. It documents the observations about custom web clients that were implemented in Elixir, Go, and Kotlin and used to evaluate web service technologies.

Image for post
Image for post

Observations based on Minimum Time per Requests

While analyzing the behavior of various web service implementations, I observed that minimum time per requests for different web clients were starkly different. So, I plotted their minimum time per request against Actix-Rust service implementation as it was one of the most performant and reliable service implementation in the previous experiment.

Image for post
Image for post

Image for post
Image for post

On the topic of “excellence as a core strategy”, the book does a great job of providing great insights and quite a bit of food for thought and practice.

As for presentation style, it uses a style almost identical to that of “The Little Big Things” by Tom Peters. While I would have preferred a bit more structure, this style made is an quick read. So, no major complains here :)

As for highlights, the layout of the book makes many text fragments highlights — literally :) Even so, I figured I’d capture some here; a few of these are quotes. …


Thundering Web Requests: Part 5

This is the fifth post in a series of posts exploring web services related technologies.

Having implemented custom web clients and evaluated web service technology using Apache Bench, I evaluated the same web service implementations using custom web clients. This post documents both server-side and client-side observations from this evaluation.

Image for post
Image for post

Setup

The setup from the previous AB experiment was used with the following changes.

Benchmarking Tools

Instead of Apache Bench, three custom web clients: HTTPoison+Elixir, Go, and Vertx-Kotlin, were used.

Each of these clients reported the time taken by each request. They also reported if a request succeeded or failed based on the error reported by the underlying web service technology. …

About

Venkatesh-Prasad Ranganath

Programming, experimenting, writing | Past: SWE, Researcher, Professor | Present: SWE

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store