---
type: blog-post
title: I am cramped
description: This post describes the ideals and values of my upcoming data platform for my homelab
draft: false
date: 2023-09-09
updates:
  - time: 2023-09-09
    description: first iteration
tags:
  - "#blog"
---

Cramped is an experiment at building a fully open source data platform built on
both new and old components. This project is heavily biased towards my ideals,
this generally means no java (even if extremely illogical in the data world, I
will dig into this in a later post) where it can be avoided, the system should
scale, but should be able to run on somewhat resource constrained environments,
like a normal developer workstation.

My primary ideals are:

- Opinionated: I want to get 80% of the way there with 20% of the effort, if
  your use-case is in the 20 remaining, then this project is not for you
- Efficient: The project should feel fast, in execution, but also to develop on.
- Minimal: The project should feel minimal, even if it has a lot of
  complexities, what is done here is trying to create actual abstractions, and
  let cross talk between components be as limited as possible
- Immutable and compose-able architecture: The architecture should feature
  immutable steps, and present a consistent api of which real applications can
  use as its bedrock. Each component should be developed in of itself, this
  doesn't mean that components shouldn't communicate, but that they will have a
  valid migration or use pattern in of itself.

## The name is "Cramped"

I like to use words that described a feature I'd like to avoid, I usually call
the project exactly what I am trying to do to avoid doing while developing it.
Sometimes it succeeds other times it doesn't but it is my ideal. For example in
World of Warcraft, I played a Warrior in which my play style would best be
described as _Coked out if its mind Squirrel_, aptly named Reckless, I gave it
that name to try to better my behavior, and actually play like a sane person.

Cramped is the feeling of being stuck in a small space, where even if you can
make room to do stuff, you feel constrained, annoyed and uncomfortable. I'd like
exactly the opposite, a platform where you feel like you can make the changes
you need, and iterate as you'd like, while having fun working on new and old
parts of the system, and generally trust that the changes you do makes sense.

## The goal

As mentioned Cramped is first and foremost an experiment. I've maintained an
existing data platform single-handily (to mixed success) since January (we're in
September now), as such I've not had the experience of actually building a data
platform from scratch, and with a lot of dependent engineers and analytics, I
can't really make as much experimentation as I'd like. Which gave rise to the
feeling associated with the project name.

I hope this project will help me work out some of the frustrations I've had with
our existing system, as well as get some knowledge on how to improve it, what is
current and best practice atm.

I am a platform engineer first and foremost, so definitely not a Data engineer
or Data Ops engineer. I am doing this for fun, but also go get some more
experience, so please go easy on me, as I work through and share my decisions.

## Approach

I will start by defining my top level architecture components, and simply start
small on developing each part. Each part should provide inside into which
decisions I am going for and why I am doing so. I will use esoteric languages
and tools. However, most of them have alternatives that may or may not be useful
for your platform.

Each piece will be developed by itself, and be deployed independently. I will
also use my own tools `cuddle` and `churn` for respectively
`Development Experience Platform` and `Deployment and Orchestration`. I don't
know why I have a fascination with *C*s... Each of these tools aren't ready to
see the light of day yet, but they're incredibly useful for my own development.

## Next

I will add a Table of Context of the different parts as we go.

- TBA
