The Problem
People were asking for a secure mechanism to exchange files. Today we are proud to announce the new open source fileserver module. As the newest component in the ecosystem of Psono it solves multiple problems:
- Files being shared by mail become too big. Usually everything above 20mb is rejected.
- Upload of files to other parties by SFTP, GPG mails and so on is too complex for normal users
- Upload of files with the help of Google Drive or Dropbox may be problematic for sensitive data.
- Protection of HR or management infos from internal employees, e.g. Fileserver admins.
- Hacker gaining access to your fileserver
- Bandwidth problems when you work in a remote office environment and everyone is downloading from your small office fileserver.
Ands that's where Psono comes into place. Psono was designed with all those problems in mind and is extremely flexible.
The Usability.
Ok, so how does it look for the user you are asking. Simple.
- A user accesses the web client
- selects the file
- clicks the "upload" button
- ... and if he wants he shares it with a right click
Thats simple enough I hope.
Under the hood
Under the hood the client does some more complex steps yet all of them are hidden from the user:
- The client asks the server for all available fileservers
- The server evaluates the request and decides according to various user attributes (permissions, IP, access rights) which shards (file locations) and fileservers are suitable
- The server creates a file upload ticket
- The client reads the size of the file
- The client announces the upload of the file and shard to the server
- The client splits the file up into chunks of a max of 128mb if the file is too big
- The client encrypts the chunk
- The client calculates the checksum of the chunk
- The client uploads the chunk to the fileserver together with the file upload ticket
- The fileserver sends the upload ticket to the server
- The server validates the ticket
- The fileserver accepts the upload of the chunk
- The client continues with the next chunk till all chunks are processed
- The client updates his datastore, ecnrypts it and sends it to the server
With this mechanism we can solve any file size problem, as files are split up in chunks. We can distribute uploads across servers even for a single file. All files are (like everything else at psono) encrypted before they leave the client and are sent to the server.
Typical Psono Fileserver Setup
The top part shows a typical Setup. A relay server (a simple reverse proxy) serves some static files (the webclient). The Webclient is served to the browser, which then calls REST API endpoints on the server. The server uses a Postgres Database to store all persistent data. The Psono Webclient and Psono Server can be considered stateless, all data is stored in the Postgres database.
If you want to attach now a fileserver. you first have to create a shard and cluster confiugration on the server, which you can then take to configure the actual fileserver. Once the fileserver boots up, it will automatically register itself and continue to announce its availability in 10 second intervals. Should a fileserver die, the server will recognize this and stop sending clients to the dead server. Once it comes back, it will automatically register again and start receiving requests from clients. In the picture above, we are storing all chunks in local storage and have a Relay server (could be the same as for the normal server, webclient in small environments) for the SSL offload.
Besides local storage, you can use various storage backends and even mix them up.
Mix of various storage backends
Currently supported (next to local storage) are Amazon S3, GCP Cloud Storage and Azure Blob Storage.
You can use multiple fileservers in order to solve for example performance problems due to a bandwidth limitation. You could have people working remote yet only a very thin internet connection in your office. Another usecase could be, that you want to protect local files even better and
- not allow internal files to be uploaded to an external storage
- allow externals to share files with internals without a different tool
A setup that solves the performance problem and gives you a better isolation could look like this:
Remote location
This setup shares all traffic of the "external" shard between sites bidirectional, allowing externals to upload files that can afterwards be downloaded by employees and visa versa.
The on-premise shar on the other hand is never synct to the remote location and stays better protected. As files in the "external" shard are only synct once, whenever they change you could have 100 employees accessing the file from this shard without the need to download it 100 times from the remote location. Or if you upload a file, you can share it with 100 externals, without the need to upload it 100 times.
The bandwidth advantage can also be used to connect remote locations with a setup similar to this one:
With two remote offices
In this setup every site has an own fileserver, and all files are shared between sites. Writes only go to a specifc shard, which allows an unidirectional sync of each shard. This setup would allow people in New York to access files without any delay.
The next big point is loadbalancing. All this is nice if you have just a few employees that do not saturate your bandwidth. If you have 1000 employees, all accessing the fileserver regulary, then this will produce bandwidth limits for a single server. Yet luckily Psono can handle this. Here are two options that are offered by Psono's fileserver for loadbalancing.
You may have already a loadbalancer that you would like to use, then your setup could look like this:
Loadbalanced fileserver with loadbalancer
In this setup a lodabalancer is used for the SSL offload. Two fileservers sitting behind are used to handle the actual requests. Both announce the same shard, so they either need to sync the shard or have some shared storage. One could use GlusterFS or a simple rsync.
The alternative would be, to allow the client to loadbalance everything and save some setup and costs. A setup without an extra loadbalancer could look like this.
Loadbalanced fileserver without loadbalancer
All those setups can be stacked, e.g. 3 clusters of 2 servers each. With each cluster serving the same shard and has a synced (or shared) storage underneath,
I hope I could give you a good impression about the capabilities of Psono's new and mighty fileserver module.
