Weekly Update (April 22nd to May 7th)

This post is the update for both the week of the 22nd and the week of the 29th.

Sync Design

During the week of the 22nd we focused exclusively on designing the sync feature. We made some important design choices that we want to share with the community:

• No hidden information — Optic doesn’t guess what you want to sync or keep the relationships in some unreadable config file. All the relationships between parts of your code appear within the code itself. Annotations are used to name the models found in your code and then set those models as the source of another part of code. When you transform code these annotations will be added automatically unless you opt-out of creating a relationship between the source of the transformation and its output.

“//name: User Model” names this section of code “User Model”

“//source: User Model -> optic:mongoose@0.1.0/createroutefromschema” defines a relationship between “User Model” and this section of code. Changes to “User Model” will trigger changes to this code.

• Sync will not happen automatically — Let’s be realistic, the last thing any of us want is a bot making unannounced changes to our code as we work. Optic maintains a dependency graph of all the key sections of code and when it sees a change that will trigger a sync it notifies the programmer. The programmer can then review Optic’s pull request, make any changes they see fit, and update the source code.
• Sync Pull Requests walk entire tree — The sync graphs Optic constructs for a project can be very complex. For instance a form might be synced with a route, which might be synced with a model definition (User Model -> User Create Route -> User Create Form). If you change the User Model, Optic will walk the entire sync graph and stage changes for each affected relationship. There are sanity checks built in that ignore circular dependencies and other invalid states that could happen when computing the patch.
• Single Project (Sorry!) — In Version 1.0 of Optic (launching this month) the sync feature will only work within the context of a single project. We’re working to make it available across multiple projects i.e. your node backend syncs with your Android Kotlin Project.

Sync Implementation

During the week of the 29th we implemented the entire Version 1.0 Sync Feature within the optic-core project (see feature branch here). Here’s a quick slideshow of sync in action:

Here we have a Model Definition for Users and a Post Route we created by transforming that model. If you calculate a sync diff on this example you get no changes

Here we’ve changed the name of the model from ‘users’ to ‘people’ and added a new field ‘age’. Before Optic you’d have to manually find and update all the code that depended on the shape of your model. Optic can generate a patch to make these updates for you without overwriting other code you wrote.

Here’s a look at the Pull Request Optic generated for this file after you changed the model

As you can see, we’ve only implemented this on the backend. Our next week of work is going to be dedicated to adding a GUI for Sync to the Optic App.

Some more thoughts on Sync

For months it’s been clear that the sync feature was going to be one of Optic’s big selling points to teams of programmers. To be completely honest I thought it was going to be a months long project to build sync and that really worried me. But the more I thought about it the easier I realized the task would be. The entire backend implementation ended up taking a little less than 8 days.

How was that possible? Well I had basically everything I needed already built. Optic can extract models from code and update source code to reflect arbitrary changes to those models — that takes care of all sync’s interactions with the raw source code. Optic also maintains a graph of all the code it recognizes in your project — a new edge type ‘DerivedFrom’ was all we needed to store sync relationships. And when it comes time to apply a patch, Arrow’s Change Evaluation interface makes it dead simple to post changes to the code base.

Optic’s core abstractions made a feature like sync a one week project (the diff function was the hardest part). Imagine what else might be possible? What else could you automate? What analytics could you provide teams about their code? What kind of visual tools could you build on top of a codebase?

Thanks for all your support! 1.0 is coming soon