Whenever I see engineers writing about their stack I am curious to learn new tools. Especially when you are running
things solo you are always looking out to learn from the experience of others.
In 2015 I was frustrated at the current level of password managers, especially that there was nothing decent for people
running private servers. I started working on Psono, an open source password manager with a decent level of encryption and
secure sharing options. Psono was never intended to run as SaaS, otherwise I'd probably have backed in support for multiple
tenants right at the beginning.
Just so you are getting a bit an overview of the scale of Psono. Psono is currently installed locally in about 70
countries, with more than 4000 installations. Psono SaaS alone is currently used by over 15.000 people, most of them using
our free community edition of Psono which runs on the same tech stack as our paid Enterprise product.
The stack of a typical Psono installation consists of a database, server, two web clients (one for regular users and one
for admins), apps for iOS and Android, browser extensions for Chrome and Firefox. A customer can install
an LDAP gateway or a fileserver on top of that.
Next to that I have a lot more infrastructures in place. Starting with the website to power the company homepage and
product page I have a license server for our enterprise product, a central ERP system to handle quotes and invoices, a
shop where people can purchase Psono SaaS and manage their installation and an authentication service with OIDC.
Through the course of the article you will see that I prefer old and boring tech.
- Python: All of the backend code is written in Python. The greatest benefits of Python are its rich ecosystem. The lack of types (alot of people consider this a drawback) is making it extremely fast to write your web service which as a startup is one of the key aspects that you should look out for in a programming language.
Python is for webservices my favorite language. (I leave out my thoughts here why not XY is better than Python. That would fill another blog article.) There is just one drawback that you have to counter and that's the possibility to ship code with all dependencies reliable which is especially crucial if people install your software locally and are asking support if they run into issues. (Speed wise, which other people might add here doesn't matter. Python is fast enough).
- Django: "It's like a super power for solo developers" (if I may quote Athony N. Simons here). Together with the ecosystem it solves nearly all problems that you may encounter. Authentication, templates, email, database migrations, totp, ... just to name a few.
- Django Rest Framework: It's currently powering all APIs and is doing a great job. The different layers (serialization, authentication, permissions, views) keep your code extremely clean.
- React: I just recently migrated the regular webclient to React. The old web client (please remember that it was started in 2015) was written in AngularJS and it was quite an endeavour to stay backward compatible with the old client e.g. all paths to the user should not change, as they might have bookmarked links to secrets in their documentation or have so-called link shares in use. Also people used custom branding with own logos and so on where paths on the system should not change.
- webpack: I wanted to mention it here, as it's currently the only part that I am unhappy with. It's slow, hair pulling, hard to learn ... the only advantage that you have is that you are never getting stuck. Any problem that you might experience, was already asked on Stackoverflow and you find a solution.
- Gatsby: Powering our static websites for the company and the product. In earlier times I used wordpress, yet the management overhead with updates and the constant fear of the website being compromised (and the potential negative publicity) made us switch to Gatsby. So far never looked back.
- Vuepress: The most decent docu management system that I could find. It compiles to static websites, looks modern and currently powers Psono's documentation
- Material UI: The main library powering the frontend. It's feature rich, plays well together with React and looks quite modern, in contrast to older Bootstrap based themes.
There are for sure countless others that would deserve to be mentioned here, yet these are the "most important" ones according to my opinion.
Even so Django supports multiple databases I was quite early on hitting the question if I want to support all of
Django's databases or only a particular one. The root for this question was that I was storing nested tree structures in
the database. So trees with multiple roots and shared branches. Querying these efficient was crucial so after exploring
all available options I decided to go with Postgre's ltree extension.
If you are planning to develop anything and run it solo, it needs to run in the cloud reliable, managed by the cloud
provider. You most probably also don't want to be vendor locked in, so nothing cloud specific, which limits your choices
- Postgres: There are two open source databases that you can choose from. MariaDB or Postgres. I was always planning to go with Maria (mostly due to Maria DB's Galera Cluster, one of these missguided "young developer needs to plan for hyper growth" stories ;D) yet due to the missing ltree extensions my hands were forced. Nowadays with all the development that goes into Postgres sharding options I never regretted that decision so far.
- Redis: Planned to be used as cache for bigger installations. Also here you can choose either Redis or Memchache. Both are excellent, yet mostly due to Redis protocol being nowadays the de facto standard for caching I was going with Redis. The only drawback that I ever experienced with Redis is it being single threaded.
Just a small overview of the infrastructure that Psono relies on:
- Gitlab: Gitlab is powering all the automatition of Psono. Noteworthy everything in Psono is automated which is one of key aspects how I can manage releases and maintain the whole Psono Ecosystem. Back in 2015 every open source project was running on github, but Gitlab was (and still is) just so mighty and featurewise superiour (with the exception of github's search functionality). Gitlab powers builds, releases, vulnerability scans of EVERY piece of software that we use.
- Cloudflare: Probably the most innovative company that I know of that influenced me over the past 10 years. It's fair to say that Cloudflare is powering the internet nowadays. It provides me with reliable nameservers, TLS, a decent CDN, and advanced features like their edge worker. The DDoS protection and WAF functionality provide a certain extra, which as a password manager is quite important. I'd love to use their SSL as a service feature to allow SaaS customers to configure their own domains, yet the paywall of Cloudflare's Enterprise level is just too hard. I am still hoping that they release it somewhere as an independent plugin that one can purchase.
- namecheap: My goto solution for domains and ssl certificates.
- Google Cloud Run: All backend servers run on Google Cloud Run, which is extremely cost-efficient and reliable when you want to spawn a fleet of services. Once you have your stack running its doing a pretty great job. Their top notch API / cli integration is an absolute plus here. There are a couple of drawbacks, like the missing support from Google or the threat that Google might decide at some point that being #3 as Cloud service provider might not be worth it and they cancel the product.
- Artifactory: Protects my build pipeline against npm outages and stores my build artifacts (which wasn't bundled in Gitlab in 2015)
- Docker Hub: Stores all docker containers and does some additional security scans and distribution of my containers.
- Mailgun: Handles all emails. In contrast to most email solution providers, Mailgun has decent logging and analysis functionality. Their support is top notch. The only problem is that their emails are sometimes being sent to the spam inbox (even with SPF, DMARC and so on all in place), yet besides that they are doing a great job.
- Poeditor: Handles all translations. The nice thing is that contributors can request access, which is crucial if you want externals to help translate your Project. Their support is awesome and you get extremely fast replies.
- Docker: One of the best decisions for the whole project. Especially since I am using Python and shipping working Python code with all Python / OS library dependencies is hard. If things are supposed to run with a customer on premise and the requirement that he can update and maintain the system. Docker allows us to do all that and gives me full control of the installed packages which is solving at least 70% of all support issues (while introducing 20% new issues as people are not used to Docker ;D)
So things may go wrong and you need to know when that happens and get some insights into the issue. That's where these tools come in handy:
- Google Cloud logging: If you are on Google your natural solution
- Sentry: My number one error report solution. Once you tried it out you will never want to miss it. It gives me the necessary insight into all errors.
- Uptimerobot: Calls the healthcheck of my various services and will alert me should anything go down. The public status page for Psono btw can be found here https://stats.psono.com/, which is only a small fraction of the services that this thing monitors for me, yet hopefully demonstrates a bit the reliability of the system.
A small overview of how my local development stack looks like:
- Windows with WSL: I am probably getting alot of heat for this, yet yes I am running on Windows. In the past years I was always having somewhere a Linux VM that I would connect to for development, yet nowadays, it's powered by WSL. I am getting the best out of both worlds.
- Pycharm / Webstorm: I am using Jetbrain IDEs. The remote development functionality is crucial for my development workflow and this "batteries included" philosophy suits me well. Sometimes I have about 20 projects open which requires a decent hardware with enough RAM. VSCode seems to be a good alternative nowadays, yet over the course of the last couple of years I got used to Jetbrains and learning something new always requires some mental strength that I am currently not willing to invest into this. I'd say everyone should use something that he is used to.
There are certain tools of course required to handle the support for people using the product:
- Freshdesk: A typical email ticket software. It helps me keep track of the various customer requests. In the earlier times it provided a decent free tier, which especially at the founding stage is crucial.
- Discord: Some people might prefer chat or require a more direct conversation / help. That's where discord comes in handy.
There are certain other tools that I wanted to mention that may be helpful for others. Especially the banks for people who want to run their business in Germany:
- Odoo: The best ERP system. It's open source edition is free and can be easily extended with python and some XML knowledge. It's customization and integration options with custom APIs are top-notch.
- Holvi: Our main bank account. In Germany, you need a bank that allows you to set up a Founder Account, in German "Gründerkonto", so a bank account that is tied to a company that is not yet completely registered and where you can transfer the initial cash which is required to register the company. A little of a chicken egg problems. And the big banks that allow that usually cost you a leg and an arm. The registration process also worked pretty flawless. The only drawback is the missing support for international payments, yet that's where the next service comes in handy.
- Wise: Our second bank account. Their fees are extremely low (e.g. no monthly fees), the setup process works flawless, they are well-connected and allow international bank transfers and transfers are quickly carried out, so you don't have to wait weeks or similar.
I guess this could be a separate blog post about how we structured user accounts, infrastructure, access rights. As this is one of the most crucial aspects to the
security of Psono I am not going into details here and try to be vague enough. We are using both Google Cloud and Azure services. Both with completely separated users and groups.
There are users dedicated for administration tasks and user accounts for the "daily doing". There are different physical and virtual systems that I use to access / support / manage / administrate Psono.
Administration tasks are only done on a separate, especially hardened physical device that are exclusively used for these tasks. Every account is protected with hardware tokens.
All systems are centrally managed and security policies in place to guarantee that all devices follow the security standards that you would usually find in banks.
This article was heavily inspired by Anthony N. Simon's article about Panelbear.