Thoughts about Programming Environments

Reflections from Advent of Code 2023

I used the 2023 Advent of Code to try out “new” languages and assess how their features would affect developer productivity. Having completed the exercise, here are my thoughts.

TIL about https://apod.nasa.gov/apod/archivepix.html. So, I added a picture from it :)

Context

To solve the Advent of Code puzzles, I used five programming languages that I do not use extensively: Clojure, Ocaml, Rust, Elixir, and Scala.

As for execution, I solve both parts of each puzzle using algorithms (preferably, single file programs), cycled through the above languages to solve the puzzles (e.g., use Clojure to solve problems from day 1, 6, 11, ...), and moved to the next puzzle only after solving or giving up on a puzzle.

My development platform was Ubuntu/Linux, and my code is available at https://github.com/rvprasad/advent-of-code-2023/.

Thoughts about Documentation

The information about language features should be easy to understand, preferably with relevant examples and links to details.

In this regard, Clojure [1, 2], Elixir [3, 4], and Rust [5, 6] documentations embody this integrated approach. In contrast, Scala and OCaml API documentation did not include examples but provided separate documentation about using various APIs.

Documented information should strive to be complete.

In the case of OCaml, opening the Stdio module at the top-level created a conflict with the In_Channel module. Likewise, upon opening the Core module at the top-level with the core_unix library as a dependency, the OCaml compiler complained that String.split_on_char should be String.split_on_chars when there was no String.split_on_chars function in the standard library. Capturing such conflicts in the documentation of components that cause such conflicts can help reduce developer toil. [Strangely, examples in the Real-World OCaml book open modules at the top-level; thanks for the free book.]

Comparative/Transitional information can lower the learning curve for folks coming from other ecosystems.

Based on names, I used the Map module OCaml when I needed something similar to a HashMap in Java. Later, I realized that I should have used Hashtbl in OCaml. With comparative/transitional information, I could have avoided some toil and been more productive. On this front, Scala provides such comparative/transitional information for Java developers.

We should widely mimic Rust’s idea of documenting compiler error codes.

The documentation for each error code of the Rust compiler contains stripped-down examples that trigger the error, along with the explanation of the error and the guidance to fix the error.

Thoughts about Programming Environments

Bootstrapping single-file programs should be a breeze.

Of the above languages, Elixir and Scala offered a zero-hassle path to such bootstrapping — create a file and feed it to the CLI tool. While Rust required the cargo tool, the tool was a breeze to use with zero overhead and additional complexity — use cargo init followed by cargo run.

Clojure offered an easy solution (not simple as it involved using namespaces, ensuring some parity between namespaces and the on-disk folder structure, and using the unnecessary deps.edn file) [7, 8]. Still, it was not apparent from the documentation. Also, options such as Leiningen and edn seemed too involved for a single-file program.

While Dune was the recommended build tool for OCaml, it was neither simple nor easy to bootstrap single-file programs. It required the creation of a project to create an executable build target. In turn, this setup generated unnecessary folders.

Build systems should adopt the zero-overhead principle.

Consequently, I think build systems should adopt (at least the first part of) the zero-overhead principle: you don’t pay for what you don’t use. Besides easing the bootstrapping of simple efforts, adopting this principle can help maintain a clutter-free local workspace and improve productivity, e.g., not stumble on unnecessary files/folders and not configure tools to ignore unnecessary files/folders.

IDE should work for single-file programs without ceremonies. Also, supported features should be reliable.

I used Visual Studio Code for all other languages except Clojure. VS Code extensions for OCaml, Rust, and Elixir worked out of the box.

The Metals plugin for Scala was a hit-and-miss. The plugin did not recognize source files in a sym-linked folder in another drive. Renaming the file from X.scala to X.sc and back to X.scala fixed the issue sometimes, while restarting the SBT server fixed the issue sometimes. Sometimes, neither fixed the problem and compiling the program in a terminal was the only way to type-check the program.

Independent of the languages, VS Code slowed down to a trickle when left open for a long time, say, overnight.

I used Vim (instead of the Calva plugin for VS Code) for Clojure. Once I passed the initial learning bump, using the Paredit mode was bliss. While additional editing and navigation commands would have been more helpful, structural editing felt natural for coding and made me wonder, “Why don’t more languages have structural editing-friendly syntax?”

Surprisingly, the Rust Rover IDE just worked with the single file program. I suspect the magic of cargo init was the reason.

Every language should offer a REPL environment.

The support for REPL in Clojure, OCaml (utop), Elixir, and Scala allowed quick experimentation, expediting the understanding of an API or a concept. While Rust lacked REPL support, the Rust playground provided similar capabilities. Given the benefits of quick experimentation, new and existing languages should have really strong reasons not to offer REPL environments.

IDEs need better ways of displaying multiple errors that occur at the same program point.

When coding in Rust, compiler errors that overlapped the same program would clutter the Editor View via multiple squiggly underlines. While each error was listed separately in the Problems View/Panel, viewing them via the Inline Zone was hard.

Do type annotations help in closed and limited contexts?

In contrast, I did not encounter this issue while coding in Clojure and Elixir (as there was no type checking). Consequently, I could easily focus on translating a solution into code without being distracted by type errors. Once the code was (syntactically) complete, I executed the code and fixed the compiler errors, which were easy to fix.

Get the solution sketch right before getting the types right.

The limitations of the lack of type annotations bit me when the number of levels of abstractions increased. However, tackling the problem by starting from higher level abstractions and drilling down alleviated the issue, more so when using the functional programming mindset.

Is translating a solution into code by starting from higher-level abstractions more amicable with the functional programming style?

Interestingly, I always took the top-down approach while coding in Clojure. Did the syntax of the language strongly influence my choice of approach?

Adopt the Hemingway writing mode (Write vs Edit) for coding.

Reducing the clutter mentioned above in IDEs can help improve developer productivity, i.e., help developers focus on the solution before fixing the plumbing related errors. While we can achieve this effect by turning off on-the-fly compilation, it will negatively affect capabilities such as auto-completion.

IDEs should provide atype-hiding mode/lens.

Like IDEs display user-provided and inferred type annotations (e.g., for anonymous functions in Rust), a type-hiding mode/lens would hide user-provided and inferred type annotations and any compiler errors (or some combination of these bits).

Miscellaneous tips and gotchas

1. Coding in Clojure and Elixir was like cutting butter with a warm knife.
2. In Clojure, use recur function [9] with loop or fn functions [10, 11] to avoid stack overflows in recursive algorithms, e.g., work-set/-list algorithms.
3. In Elixir, use IO.inspect(x) to output debugging messages. To output multiple values in a single invocation, wrap the values in a list, i.e., IO.inspect([x, y]).
4. In Elixir, the pipeline operator |> provides the piped value as the first argument to the invoked function. In contrast, the piped value is the last argument to the invoked function in most other languages that offer a similar operator, e.g., OCaml, F#.
5. Functions cannot be defined at the top level of the Elixir interactive shell, iex. Instead, functions should be defined in modules, which makes the REPL experience a bit lacking.
6. While writing Rust code in a plain old editor, use cargo watch -x build -w src to get instant yet non-interfering feedback.
7. The treatment of ownership of primitive types in Rust can be counterintuitive compared to other languages.
8. Using Rust standard libraries in a functional style requires a bit of adaptation, which may be solely due to how I used Rust. Since all types do not implement the Copy trait (which does implicit copying), processing references to and iterators of collections of different types requires tracking if the element types implement the Copy trait and use copied() or cloned().
9. The Format module can help write debugging messages involving custom data types in OCaml. Of course, it requires the definition of pretty printers for custom data types, which is often simple.

Caveats and Requests

1. My observations may be incorrect as I may not have missed out on existing resources. So, if you spot such omissions, please share pointers to relevant resources.
2. The above observations and thoughts are based on one instance and likely heavily influenced by my background and experience; hence, they may not be general. So, if you have a different take on these observations/thoughts from a different perspective, please share them via a comment.
3. If you know of efforts tackling similar or related ideas, please share pointers to them.

I hope you found something useful in this post :) And, thanks in advance for any feedback or pointers that you may share!