Taking Notes

Several years ago I discovered the power of keeping a work journal and it has helped me tremendously in my job and in my personal life. It seems obvious, but the act of writing things down as you go through the day is both liberating and empowering. All too often I would work my way through a technical problem or finish up a big project and say, “Well, I’m glad that’s done.” and then promptly forget most of the nitty-gritty details that were the key to solving the issue or completing the project. I would tell myself, “that was so difficult and the completion so satisfying that there’s no way I’ll forget that.” But I almost always did. Diligently keeping a work journal has helped me be able to recall those little details quickly the next time a similar projet or problem pops up. I won’t pretend that I’m an expert at it, or even that I’m diligent about it, but I’ve found that even writing anything about the work you did helps you recall more details later on.

The Journey

At first I relied on whatever ticketing system we used at work to keep track of things I did and/or solutions. However, most ticketing systems end up getting in the way more often than not when you’re trying to research solutions to problems you yourself, or other people may have worked on in the past. Ticketing systems are mostly just metric tools for managers to show how much work is being done. Also, there are things I may want to make a note of that really shouldn’t go in a ticketing system - stuff I don’t want others to see, or that they shouldn’t see.

RedNotebook

The first application I tried to keep track of work notes was RedNotebook. It was a simple little app that had it’s own little markup language similar to Markdown. My favorite feature of it was the calendar view in the top left corner. You could easily click on any day and the note field on the right would populate with your journal entry for that day. It was great if you knew when you worked on something and could jump right to it. It also maintained a tag cloud so you could easily find notes with similar tags. There was support for templates and I created some small templates to manage the kinds of things I was dealing with on a daily basis (e.g. Jira tickets, project notes, communications, etc.).

Unfortunately, as I used the app more and more it was starting to show some cracks. The most frustrating thing for me was the markup language. I could never quite get the hang of it, despite it’s similarities to Markdown. Another thing that bothered me was the storage format. ALL your notes are stored as one giant JSON file. While I never really had any trouble with corruption, or load times, it just seemed fragile to me.

Evernote

Yes, I hate to admit it, but for a while I actually used Evernote. I’m not proud of it. I liked the easy formatting and the ability to send notes via email, or just clip entire websites from my browser straight into an Evernote note. Since there is (still) no official Linux client for Evernote it was a constant battle to use on my Linux work or personal machines. The web interface was not quite as easy to use as the (Windows) desktop client. Trying to run the Windows client under Wine emulation was really hit-or-miss. If you managed to get it to install it would randomly just stop working, or the fonts would suddenly be all messed up. I gave up on that and tried to use just the opensource Nixnote client. It was much more stable, but it didn’t handle the Evernote formatting very well and often my notes would just look like a horrible mess. I let my subscription expire when I decided that I needed something both simpler and more powerful.

Markdown + Git

I don’t know what I was thinking, but for a brief while I thought the best solution was notes in Markdown stored in a git repo that I could sync around to my various machines and phones. For simple text notes it’s not a bad solution. It can get tricky if you edit the same note on multiple devices, but that’s what git is good at. What this method is NOT good at is keeping track of images or other attachments to a note. Storing binary data in git is just wrong, so I was hesitant to put images and such in the repository. However, some notes just needed a picture or diagram to go with them to explain the concepts.

The system was manageable from desktop machines or laptops, but phones and tablets were a different story. First of all, at the time there were no decent git clients for Android. I had to use command-line git from a Termux shell. While it was usable, it was not very fast or convenient when you’re trying to take a few quick notes on your phone.

What I really was looking for was something with the Ease of Evernote and RedNotebook but with the power of Markdown and git. I also wanted something that was self-hosted and free from some corporation’s control.

The solution I found was Joplin.

Joplin

Joplin is an Electron-based app written in (mostly) javascript. It is far from what I would call “light-weight”, but it is reasonable in terms of memory and CPU usage. It is open source and is actively developed. It has a web-clipper component with a browser plugin that can store markdown versions of a web page directly into your note database (pictures and all). It does a good job of managing images and attachments.

For formatting, it uses Github Markdown syntax so it is easy to work with and the format is widely used. The app preferences allow you to enable other “dialects” or sub-features of Markdown, like Mermaid graphs, or advanced table syntax. You can even turn on MathML for your math notes. Recently, a plugin system has been added that allows for even more customization and expand-ability.

Joplin supports several methods of synchronization, including local filesystem, Dropbox, Onedrive, Nextcloud, WebDAV, AWS S3, or the new Joplin Server. Joplin also supports end-to-end encryption. This means that if I’m using Dropbox as my synchronization target, the data that is written there is encrypted both “in-flight” and “on-disk” at the target. Ihe encryption is pretty fast, even on a low-end android phone. I didn’t fully understand how the encryption worked the first time I set it up. I had imported a few hundred notes from my Evernote export into the desktop application and I then set an encryption key on it. I then went to my phone and set it up there. I thought you needed a separate key for each device, but it turns out you do not. By setting a new key on my phone Joplin was having to encrypt and decrypt twice every note in the database. It did not go well on my older android Note 4 that I used for work. Anyway, once I figured out how it was supposed to work everything syncs and encrypts/decrypts quickly.

The phone/tablet app is a little clunky, but it works well. Best of all, it seamlessly manages the attachments, synchronization, and encryption so I can focus on taking notes.

Joplin is not truly what you could consider “self-hosted” - yet. You can configure various locations to sync your notes to, as mentioned above, but it treats them just as a storage location. It handles locking well so you don’t have 2 clients (e.g. your PC and your phone) trying to make changes at the same time. The developers of Joplin are working on a new back-end called Joplin-server, which is supposed to be a better option in that it will be much faster and it can be truly self-hosted. I’m looking forward to trying it out.

So that’s where I’m at. I’m using Joplin for both work and personal note taking. It has saved my butt multiple times at work when I needed to perform a task that we only need to run once a year and nobody bothered to put it in a wiki page because “I’ll remember how to do that next year”.

Next post I’ll write a bit more about how I use Joplin and maybe some templates that I’ve developed to keep track of common things.