Rapid iteration

In software development

2023-11-29

Introduction

• Idea
  • Works in category theory
  • Functorial semantics
  • One syntax, many semantics
• Proposal
  • Implement in Typescript
  • With code generators as functors

What is rapid iteration?

• What is iteration?
• How an iteration becomes rapid?

Iteration

• Iteration = Changing = Rewriting = Refactoring
  • Add, remove, rewrite a feature
  • Change a 3rd party provider
  • Fix a complex bug, etc.
• Writers rewrite a novel around 10 times ¹
  • To achieve a canonical, stable form
• Perhaps software needs the same treatment

1. How many times to rewrite a novel? (Google search)

Refactoring ¹

1. Refactoring has a price. Not refactoring has a cost. Either way, you pay.

Example iteration

For an e-commerce store

1. Launch MVP
2. Change the authorization provider
3. Add sorting, filtering to products

Rapid iteration

1. Launch MVP
   1. Prebuild the app from existing components
2. Change the authorization provider
   1. Have a generic authorization interface
   2. Adapt the new provider to the interface
   3. Write no more code
3. Add sorting, filtering to products
   1. Have a list component
   2. Which already has sorting, filtering built in
   3. You just need to enable these features

Is rapid iteration possible?

• Tried for ~20 years, with no success
  • Using classic programming languages
  • And paradigms
• Tedious manual work
  • Abandoned the idea, as impossible
  • Did slow, manual refactoring
• Then found Category Theory

Category theory ¹

• How to design abstractions
• Which compose well (the natural way)
• Yet remain extensible (iterable)

1. Scientific and software engineering examples of applied category theory

Category theory ¹

• It’s complex …
• But the basics are simple
• There are categories
• And functors

1. Category theory

Category

• Deals with objects and their relationships
  • Objects: X, Y, Z
  • Morphisms: f, g
  • Composition of morphisms: ∘
• Based on certain axioms:
  • Identity: 1b ∘ f = f = f ∘ 1a
    
  • Associativity: h ∘ (g ∘ f) = (h ∘ g) ∘ f

Categories are everywhere

Category theory	Software module	Algebraic structure 1	Concept-based design 2
Objects	Types, data structures	Domain	Generic types
Morphism	Functions	Operations	Actions
Axioms	Tests	Axioms	Operational principles

1. Algebraic structure

2. The Essence of Software, MIT CSAIL

Concept as category ¹

1. The Essence of Software, MIT CSAIL

Functors

• They are not everywhere
• As far as I know

Functor ¹

• A mapping between categories
• Associates each
  • Object
  • Morphism
  • From a category to another
• Preserves the axioms

1. Functor

Functorial semantics ¹

“One syntax, many semantics”

• Category = Syntax
• Functor 1 = Semantics 1
• Functor 2 = Semantics 2
• Functor n = Semantics n

1. AlgebraicJulia: a compositional approach to technical computing

Applied Category Theory / AlgebraicJulia

• There is a main category
  • Describes the problem
  • This is the syntax
• There are additional categories
  • They add computational content, like
    • Execution, verification, visualization
  • These are the semantics
• Functors
  • Auto-map the main category
  • To the additional categories

Rapid iteration with category theory

• It’s very, very easy
• Watch the functors !!!

• In the following diagrams:
  • A dotted arrow
    • Is a functor
    • Is an automatic task
    • Performed by the system
    • In the background
  • A standard arrow
    • Is a manual task
    • Performed by the developer

\[\begin{gather*} \dashrightarrow\text{Automatic / Functor}\\ \rightarrow\text{Manual task} \end{gather*}\]

Manual iteration ¹

All tasks are manual

1. Scientific and software engineering examples of applied category theory

Rapid iteration ¹

Just a few manual tasks …

1. Scientific and software engineering examples of applied category theory

Manual vs. rapid iteration example

Python, classical, Write code

"""Migrate a Sim to a Sim2"""
def migrate_sim(sim::Sim):
  init_sys = System2(sim.cell, sim.atoms)
  final_sys = unknown(init_sys.cell.pbc)
  return Sim2(sim.calc, init_sys, final_sys)

for old_sim in get_sims(old_db_cxn):
  insert_sim(migrate_sim(old_sim), new_db_cxn)

new_query = "SELECT S1.pth, S2.pth FROM ..."

Algebraic Julia, categorical, Declare the change

# Migrate old data
new_simulation_db = Σ(F,constraints)(sims)

# Migrate old query, Q
new_Q = Σ(F,constraints)(Q)

In Category theory / Algebraic Julia

• There are very few manual tasks
  • Only declarations
  • Of components, compositions, changes
  • Initial values, input props
• There are lots of automatic tasks
  • Executed instantly
  • In the background
  • By the system / functors

In Category theory / Algebraic Julia

• The focus is on
  • Conceptual understanding
  • Modelling the world
  • Creating / declaring categories
• All the rest …
  • Like execution, verification, visualization
  • Is handled by functors
  • In an automatic way

Category theory / Algebraic Julia

• Academic software
• Not ready for the industry yet

Is rapid iteration possible without ACT?

• Formal methods ¹
  • Yes, model is math
  • Expensive ❌
  • Not for everyday software
• Semi-formal methods ❓
  • Maybe, model is semi-data (DSL)
  • Affordable
• Informal methods ❌
  • No, model is code

1. Formalism, correctness, cost

How to replicate rapid iterations from ACT?

• First we need a category
  • Or something similar
  • Generic, interface-like, with declarative syntax
• Then a functor
  • Which transforms the interface
  • Into an implementation
  • “It gives a computational meaning, content”
• Therefore a rapid iteration is:
  • Declare the changes in the interface
  • (Re)run the functor

Example

An algebraic structure in Typescript ready for rapid iteration

The List algebraic structure

• It’s a generic interface / type
• Equipped with a rapid iteration operation
• Which is a code generator

type List[T] {
  domain: {
    items: T[]
    currentItem: T
  }
  operations: {
    // The rapid iteration operation
    generateListImplementation: (config) => {...}
    // List specific operations
    addItem: () => {...}
    removeItem:() => {...}
    sortItems: () => {...}
    filterItems: () => {...}
    ...
  }
  axioms: {
    whenAddItem: "The created item becomes the currentItem"
    ...
  }
}

The code generator

• Keeps the model / interface intact
• Implements the model
• Based on configuration, options
  • Declare the changes

function generateListImplementation(config) {
  const { name, options } = config
  const { operations, formLibrary } = options
  /**
   * 1. Generate a list named "name"
   * 2. Make available the operations from "operations"
   * 3. Use the "formLibrary" 
  */
}

Usage

• You can generate on the fly …
• … different implementations of the same interface

generateListImplementation(
  {
    name: "Menu",
    operations: ['list', 'click']
    formLibrary: ""
  }
)
generateListImplementation(
  {
    name: "Articles",
    operations: ['list', 'click', 'addItem', 'removeItem', ...]
    formLibrary: "uniforms"
  }

The result

• The code generator
• Implements the models
• Into /app

├── algebraic-structures
│   ├── forms
│   │   ├── formik
│   │   │   └── generateFormik
│   │   └── uniforms
│   │       └── generateUniforms
│   └── list
│       └── generateList
└── app
    ├── articles
    │   ├── AddArticle
    │   ├── clickOnArticle
    │   ├── ListArticles
    │   └── RemoveArticle
    └── menu
        ├── clickOnMenuItem
        └── ListMenuItems

Example iteration

• Just update the config
  • Declare the changes
• And run again the generator

Iteration 0

generateListImplementation(
  {
    name: "Articles",
    operations: ['list', 'click', 'addItem', 'removeItem', ...]
    formLibrary: "uniforms"
  }

Iteration 1

generateListImplementation(
  {
    name: "Articles",
    operations: ['list', 'click', 'sortItems', 'filterItems']
    formLibrary: "Formik"
  }

The result of an iteration

• The new implementation of the model
• The old implementation is kept for
  • History
  • Manual merge, if needed

── app
    ├── articles
    │   ├── clickOnArticle
    │   ├── FilterArticles
    │   ├── ListArticles
    │   └── SortArticles
    ├── articles.1
    │   ├── AddArticle
    │   ├── clickOnArticle
    │   ├── ListArticles
    │   └── RemoveArticle

Summary

• Rapid iteration is possible
  • With a new technique
  • Inspired by Applied Category Theory
• The problem, the model, the concept
  • Is a generic interface
  • Has a declarative syntax
  • Changes, iterations go here
• The solution
  • Implements an interface
  • Automatically, via a code generator
  • Doesn’t cares about changes, iterations

Thank you!

• hi@osequi.com