experience

In designing this application, a few goals had to be met:

1. Ad-Hoc creation product variants due to the 2nd hand nature of items being bought and sold.
2. End-to-End tracking of an item's lifecycle such as storage location, repair status, upgrade status and item value.
3. Complete real-time synchronization without UI refresh between all users when changes are made.

To address the real-time data challenges, we had to maintain a synchronized object model across all clients using an event-sourced diffing algorithm. The dynamic nature of incoming data lead us to use a NoSQL solution to store our data as compared to SQL like the previous iteration. As everything is computed client-side, heavy optimizations had to be made to keep the page render times under control as the synced states could generate up to a few hundred megabytes of JSON data.

Writing and using starfuel brought a few concerns into scope:

1. A dependency requirement on leveldb for a small scale database which introduced deployment failures due to compilation errors.
2. Performance issues on large amounts of data.

Using the ideas developed during the creation of starfuel, I solved the issue of deployment and performance by picking the rust programming language. The result is a stable, performant, and portable successor to starfuel. This project is currently in development and slated to take over the starfuel name and become open source in the future.

This was developed due to my goto validation library, superstruct, being unmaintained and buggy for a period of time. It is designed to allow the following validation patterns:

1. Fully recursively composable validators.
2. Custom validation.
3. Transformation before validation.
4. Transformation after validation.
5. Type interface extraction from schema.

I've used multiple command line tools like oclif, yargs, and commander. However they all had a few common issues:

1. Unwieldy to use.
2. Poor type system and guarantees.
3. Lack of nested commands support.

This cli framework was created to solve these issues with the following patterns:

1. Using the builder pattern to generate complete types for the CLI action.
2. Using react to render an interactive help menu on the command line.
3. Dynamic nested folder command detection as well as easily composable child commands.
4. Explicitly required command parsing type validation.

We used nebula for the linking of local servers to the cloud when performing cloud bursting computation. The dynamic nature of this meant that we had to develop a way to dynamically sign and issue certificates to the cloud computers across the web without a pre-established secure network. Both the client and server were completely written in a functional reactive pattern for stability. The end product is a robust and secure client-server model for issuing and retrieving certificates.

Databases usually create these few pain points:

1. Schemas (SQL) defined separately from the code using it, requiring defining the schema a second time in code.
2. Dependence on external schema compilers.
3. In language defined schema lacking intellisense and type checking.
4. Usage of unwieldy ORMs that create more grief and pollute the code than it streamlines the experience.
5. Requiring the deployment and maintenance of a separate database container or server.
6. Lack of validation, schema and guarantees when storing data (NoSQL).
7. Lack of concurrency guarantees or poor documentation on how to handle issues such as dependent data changing after read, while in a transaction.

All these made the simple job of storing data for small projects extremely tedious. I needed a way to completely define the database from within TypeScript, complete with validation guards and migration without turning to any external tools, languages, or database deployments. Thus, starfuel was written to solve all these problems, requiring the bare minimum amount of code to define and use a database schema with working intellisense and full relational mapping guarantees. The way I solved concurrency issues is by using an event sourcing pattern and diffing each change against its base transaction before commit.

This was developed to be used with starfuel. It generates an oplog for all changes to an object and allows access of each historical state. This is done transparently using recursive ES6 proxy declaration.

I had a need for bi-directional communication and API validation. Existing solutions had a few issues:

1. They were not bi-directional (Koa, Express, Hapi).
2. They were not geared to easily define different routes (socket.io).
3. They offered no way to enforce type guarantees on the server and client.

Using this, we can define the schema once to be shared among the server and client. The client will automatically generate a connection rxjs interface to the server with all the proper types. The server will generate intellisense hints for all required API routes allowing the IDE to autocomplete the API function bodies.

Synchronizing large amounts of data between clients and servers was always a difficult, tedious, and error-prone process. Using this, we define the API schema once and provide updated and changed hooks, as well as dependency change tracking so that any change in data is immediately pushed to clients. All this was done natively in TypeScript without the reliance on external tools like GraphQL schema compilers.

With a root definition of a starfuel schema and diaract API, an API interface is created with an autoinjected database. All real-time changes in the database are automatically tracked and reflected through the API without any manual change detection setup.

A proof of concept that was very briefly used in production. Leveraging the widespread availability of the Microsoft Office Suite we developed a SQL database application to manage a second-hand laptop trading business' inventory. The reason Access was chosen was due to the extreme ease of deployment: All you needed was a shared CIFS network share, and you had a real-time synchronized database.

The application is currently being used across all camps.

The application is currently being used across all camps.