A step-by-step tutorial on configuring and running the mcp-playbook server, including GitHub token creation and Docker authentication for ghcr.io.

Setting up the MCP playbook server

A detailed look into the mcp-playbook's internal code flow, data handling for chat logs and prompts, and interactions with GitHub repositories like prompt-db and prompt-log.

MCP playbook code and data flow

Exploring the motivations behind the MCP Playbook, its role in establishing a dynamic runbook, and new patterns for capturing AI-generated knowledge like chat logs and prompts.

MCP Playbook: Automating Knowledge from AI Interactions to a Living Runbook

How data flows through the MCP Playbook server, from MCP client requests to tool execution and interactions with the file system and GitHub.

Data flow in MCP Playbook

An index and introduction to the mcp-playbook server documentation series, covering its purpose, setup, and operational flows.

Understanding and using the MCP playbook

Sharing and propogating team knowledge and practices through LLM and MCP automation.

MCP Playbook Mastery

How data flows through Brainery's second brain system, from Discord, Memo Blog, and GitHub sources to TimescaleDB via MCP, with an LLM-powered interface for natural language queries.

Data flow in Brainery

A Request for Comments (RFC) detailing a comprehensive specification for implementing OAuth 2.1 authorization within the Model Context Protocol (MCP) when using Server-Sent Events (SSE) as the transport mechanism, ensuring vendor-neutral security for AI-to-tool communication.

MCP authorization over SSE

Explore how we solved the catastrophic forgetting problem in LLMs by building an external knowledge system that mimics learning through context manipulation, without touching the model's parameters.

Build a brain that never forgets

RFC proposing a novel approach to track concept evolution using TimescaleDB with pgvector/pgvectorscale, enabling historical semantic analysis and preventing catastrophic forgetting in continual learning systems.

Append-only concept embedding log

A migration-less, append-only database design using TimescaleDB's hypertables and continuous aggregates to capture observational data and detect emerging patterns through LLM analysis.

Database design and philosophy

Building systems that actually learn and get smarter over time is hard, facing challenges with traditional databases and LLMs. This post outlines an approach to building a true second brain that continuously learns, generates insight, and remembers it through an append-only log and flexible data structures.

Building use-cases

This post explains the multi-stage process of transforming raw data into structured information, and then synthesizing that information into persistent insight and knowledge within a true second brain, leveraging tools like TimescaleDB and LLMs.

Promoting raw data to insight

Exploring the core motivations and principles behind building a personal knowledge management system, a true second brain.

Continual learning DIKW pyramid

The official Brainery build-log. We're detailing our process for creating a 'second brain' designed for continuous learning and insight generation. Covers motivation, system design, and practical data-to-wisdom strategies.

§ Brainery 🧠

This guide explores implementing robust authorization for Model Context Protocol (MCP) over Server-Sent Events (SSE) transport, providing a standardized framework for secure AI-to-tool communication while maintaining vendor independence.

Securing your remote MCP servers

A comprehensive guide to implementing granular access control for Model Context Protocol (MCP) servers, allowing organizations to securely expose tool capabilities based on user roles and permissions while maintaining data privacy.

Tool-level security for remote MCP servers

AI agents, talent shifts, & market dynamics: Explore tech trends in AI, blockchain, & software engineering. Discover key insights & analysis.

20242025

Forward engineering Feb 2025

A case study exploring how we built an AI-powered travel assistant using Ruby and AWS Bedrock, demonstrating how choosing the right tools over popular choices led to a more robust and maintainable solution. This study examines our approach to integrating AI capabilities within existing Ruby infrastructure while maintaining enterprise security standards.

AI-powered Ruby travel assistant

An in-depth look at Dwarves' monthly Project Reports system - a lean, efficient system that transforms communication data into actionable intelligence for Operations teams. This case study explores how we orchestrate multiple data streams into comprehensive project insights while maintaining enterprise-grade security and cost efficiency.

Project reports system: a case study

Project operations copilots is a suite of AI assistants designed to streamline project management and reporting processes. These copilots, built using Dify and custom implementations, help teams generate insightful project reports, create comprehensive project charters, and facilitate smooth project handovers. This approach represents a shift towards AI-assisted project management, enhancing team efficiency and project success rates.

Project operations copilots

Team copilots is a suite of AI assistants designed to enhance team collaboration and productivity. These copilots, built using Dify and custom implementations, help teams manage tasks, facilitate communication, and improve overall efficiency. This approach represents a shift towards AI-assisted team management, enhancing team performance and project success rates.

Team copilots

An exploration of natural language to database query systems using MongoDB, examining system prompts and implications for data engineering and agentic workflows.

Natural language to database queries: Text-to-MongoDB

Explore the October 2024 market report: chatbots as productivity essentials with future potential, AI's role despite investment bubble concerns, and Python's rise in modern tech and AI. Learn why coding tools require strong practices, why companies seek versatile engineers, and how Big Tech's job market favors senior talent, posing challenges for juniors.

October 2024

Market report October 2024

Learn how to transition from application development to data engineering by building a modern project reporting system. Discover essential mindset shifts, best practices, and hands-on look using tools like DuckDB, and Modal. Master system-level architecture and data pipeline design for scalable enterprise solutions.

A Lens to Modern Data Engineering

Building a data-driven project reporting system: A lens into modern data engineering

Our visit to DZS Media revealed surprising parallels between film production and software development. Their meticulous planning, focus on getting things right the first time, and expertise-driven approach offers valuable insights for our engineering practices.

DZS Media

#2 Film production lessons for software engineering

How we support, contribute to, and reward open source development in our community

☀️ Open source

In this month's market report, we've covered the hype and skepticism around OpenAI's reasoning models, the quiet rise of lightweight AI, the misuse of serverless, and the shakeup in web frameworks.

September 2024

Market report September 2024

A guide on how to check in at the office and claim ICY token rewards.

Office check-in process for earning ICY

In Q3/2024 Forward Engineering, we explore new tech stacks, engineering excellence, and key trends from the tech market over the past three months, with a focus on AI. Learn about tools like Dify, LangGraph, RAG, and more as we share our experiments and insights. Whether you’re interested in AI-driven workflows or tech market trends like the surge in AI startups and hiring demand.

Forward engineering September 2024

Journey of Thought Prompting is an innovative technique that leverages AI to craft more effective prompts for large language models. This approach uses the analytical capabilities of AI assistants to help users create detailed system prompts, fill in missing details, and iteratively refine their prompt engineering skills. It represents a shift towards collaborative AI interaction, enhancing problem-solving capabilities for businesses and engineers.

Journey of thought prompting: Harnessing AI to craft better prompts

Discover how the Dwarves community fosters a culture of continuous learning and knowledge sharing. With a monthly reward pool of up to 2500 ICY, contributors are recognized for sharing valuable insights, especially in areas like AI/LLM, Golang, and more. Get involved and grow with us.

Record and reward sharing at Dwarves

Explore the August 2024 market report to discover how AI and LLM tools like Cursor, Claude 3.5 Sonnet, and Amazon Q are transforming software development. Learn about the shift from generative UI to generative apps, the impact of prompt caching by Anthropic, and the rise of structured outputs in AI.

August 2024

Market report August 2024

In July's market report, we explore Meta's Llama 3.1 and its competitive edge in AI, OpenAI's fine-tuning for GPT-4o-mini, and Claude Sonnet 3.5's rapid development improvements. We also highlight advancements in sports analytics, multilingual text-to-speech, cybersecurity challenges, and trends in software engineering, including "boring" technology and evolving programming language salaries

July 2024

Market report July 2024

Explore how we developed a fully local search engine for our memo website using DuckDB-wasm, Transformers.js, and Alpine.js. Learn about hybrid search techniques, real-time embeddings, and performance optimizations that deliver fast and accurate results without compromising on privacy or requiring server-side processing.

Building a local search engine for our Memo website

Discover how managed LLMOps platforms like Dify streamline the development of AI-powered internal tools. Learn about the challenges of integrating LLMs, the benefits of managed solutions, and real-world examples of efficient AI tool creation. Perfect for businesses looking to enhance productivity with AI without extensive technical expertise.

Streamlining internal tool development with managed LLMOps: A Dify case study

Introducing the OGIF Memo Summarizer, a chatbot we developed using Dify in collaboration with @innno_. This tool transcribes YouTube videos and extracts key points from our Oh God It’s Friday (OGIF) sessions. By providing both short and detailed summaries in English and Vietnamese, it significantly enhances our ability to review and reference the diverse knowledge shared every Friday.

How we crafted the OGIF summarizer bot to streamline weekly knowledge-sharing

An overview of RAPTOR, a novel approach for improving retrieval-augmented language models for long documents using hierarchical tree summaries.

RAPTOR: Tree-based retrieval for language models

In May's market report, we explore GPT-4o's advancements, the rising demand for veteran programmers, the transformative power of prescriptive AI, React 19's impact on network gaps, the growing complexity of GraphQL codebases, and Bend, a new parallel programming language.

May 2024

Market report may 2024

In april’s market report, we explore the evolving landscape of Meta's Llama 3, LLM, the shift to edge computing, the new Bun 1.1, the trend of copy-pastes UI.

April 2024

Market report April 2024

Our third office hours community discussion, where we'll cover diverse topics ranging from the current state of the dwarf community and their experiences with devbox, insights on and continuation of liquidity markets, advancements in generative AI technology, to fun events like a lucky draw for icy. The goal is to foster learning by sharing weekly topics suggested through tags, encouraging collaborative growth among our members.

#3 Generative AI, Tokenomics, and Finance Talks

OGIF office hours #3 - generative AI, tokenomics, and finance talks

Our second Office Hours. Join the community to exchange knowledge and insights on diverse topics, including Docker alternatives with Nix, security practices and origin stories of our standards, financial discussions on liquidity, company updates, and icy draws.

#2 Devbox as the new Docker, Security Standards, and Understanding Liquidity

OGIF office hours #2: Devbox as the new Docker, security standards, and understanding liquidity

Our first ever Office Hours in our series of OGIFs. Our first day to exchange knowledge and insights on topics and projects we're working on and tools we're using for our internal work and clients.

#1 Markdown Presentations, Research Pipeline, Screenshots How-to

OGIF office hours #1: markdown presentations, research content pipeline, and professional screenshots

In this month’s market report, we explore the evolving landscape of artificial intelligence, API techniques, toolings, programming languages, and the intersection of design and engineering. Our insights shed light on the dynamic shifts and emerging trends across these domains, highlighting the advancements that are shaping the future of technology and software development.

March 2024

Market report March 2024

Our Discord isn't just another company server; it's a place for both our team and the wider developer community. This guide explains how it works and where you fit in.

Discord

Dwarves network discord

This month has been a continuation of AI and an upwards trend on FinOps and CyberSecurity. On top of those domains, there has been smaller, but very notable trends on the growth of Software as a Service (SaaS) or MicroSaaS' have rebound and created significant growth for serverless services, as a means to avoid startup and initial running costs. Likewise, AI has been changing how we work and how we design systems to enrich workflows and create more naturally interactive apps.

February 2024

Market report February 2024

Here are some of the few trends we are seeing across the industry, in our projects, as well as our community. This market report isn't exhaustive, at least just yet, with certain trends not yet listed such as Blockchain, data management, DevEx, etc. We see a lot of promising trends and we hope that understanding what engineers and firms are becoming more passionate about that it will help us create a foundation of ideas and knowledge.

January 2024

Market report January 2024

The Dwarves bounty program is the means through which both company peeps and the community can contribute to our daily activities. This includes tasks like building internal tools, engaging in new technology research and development, or sharing knowledge session

👾 Open bounties

Our fifth weekly catchup to kick off the new year! We organize an agenda the day beforehand and consolidate our findings, notable research progress, possibly new tech, insight or technique everyone should know.

Labs weekly catchup #5

December 2023

Market report December 2023

Our fourth weekly catchup. We organize an agenda the day beforehand and consolidate our findings, notable research progress, possibly new tech, insight or technique everyone should know.

Labs weekly catchup #4

We're a team combining tech skills, problem-solving, and clear communication. We help businesses overcome challenges by finding root causes, creating practical solutions, and working closely with clients to implement them effectively.

Consulting team

Our third weekly catchup. We organize an agenda the day beforehand and consolidate our findings, notable research progress, possibly new tech, insight or technique everyone should know.

Labs weekly catchup #3

Our second weekly catchup. We organize an agenda the day beforehand and consolidate our findings, notable research progress, possibly new tech, insight or technique everyone should know.

Labs weekly catchup #2

A collection of notes for everything we do and operate at Dwarves. This is where we keep our internal notes.

Home

Sign up for our newsletter to stay up-to-date on our latest news, tips, and updates. We'll deliver valuable content straight to your inbox, keeping you informed and engaged with stuff happening at Dwarves.

Changelog

OGIF (Oh God It's Friday) is our weekly casual office hours meeting, where team members unwind, share updates, and connect in a relaxed environment at the end of each work week.

OGIF - Oh God it's friday

Looking for a well-rounded backend engineer with experience in shipping web applications to production, CI/CD with docker centric workflow, unit testing, performance and scaling, etc. with Go/Elixir/Rust.

Backend engineer, Go/Elixir/Rust

A collection of both our internal and external events, including the things we do with the Labs team, Consulting team, Operations, team, and the community.

Digest

When we conduct an estimation, it is recommended to abandon the transitional “exact hours” assessment method, instead, use the story point based on the Fibonacci number (1, 2, 3, 5, 8, 13, 21…). The number expresses an estimation of the overall effort required to fully implement a backlog item or any piece of work.

Estimation guidelines

Our first weekly catchup. We organize an agenda the day beforehand and consolidate our findings, notable research progress, possibly new tech, insight or technique everyone should know.

Labs weekly catchup #1

This is our Labs team homepage, where we list out the latest advances in our engineering team, our publications, events & workshops, as well as frequently asked questions on who and what team labs are.

Labs team

We have focused on revamping how we learn, familiarize, and work with tech. We are excited to announce that we have updated our Forward Engineering to better reflect the feedback and insights from our Labs team, Operations Team, and Consulting Team.

November 2023

Forward engineering November 2023

DuckDB is a great starting point for modern stacks that don't need to worry about scalability, but want to derive hard analytics from the data that they have. This meeting will do a 3-part demo of DuckDB (15 minutes total): • Consolidate NYC taxi data in parallel (through map-reduce) over HTTPS • Aggregate IBM AML transactions (Anti-Money Laundering) • Some examples of how it is used to consolidate and do basic analytics on note.d and log.console.so

DuckDB demo and showcase

Explores using Golang and the Pion WebRTC library to build high-performance, scalable, low-latency video streaming infrastructure. Covers WebRTC basics (ICE, STUN, TURN), Go's concurrency benefits, Pion's native Go implementation, and a real-world multi-stream security monitoring use case.

Leveraging Golang and WebRTC for high-performance video streaming

As we step into the exciting opportunities of October, we are at a crucial point in the ever-changing world of technology. Our adventure until now has been for the most part very exciting, with a strong focus on 'Full-stack engineering’ Now, we shift our attention to a topic that is prolific and always true: Open-source.

October 2023

Forward engineering October 2023

Structured Query Language (SQL) is widely used for managing data in relational database management systems (RDBMS). In the context of SQL, the optimization of database queries forms the foundation for efficient data retrieval, providing quicker response times and increasing overall database performance. One key strategy to optimize SQL queries involves the concept of "Sargable" queries...

SQL saragable queries and their impact on database performance

Sequential read/write operations involve accessing data in a continuous, linear manner. This typically occurs when transferring large files or accessing a large file on the drive. In sequential read/write operations, the drive can read or write data from a series of blocks, allowing for faster performance...

Sql and how it relates to disk reads and writes

Our adventure until now has been incredibly thrilling, with a strong focus on 'LLM in Production.' Now, we shift our attention to a topic that is always important and never outdated: **Fullstack Engineering**.

August 2023

Forward engineering August 2023

A comprehensive guide outlining the journey for engineers developing applications on top of Large Language Models (LLMs), covering key stages from understanding AI basics to fine-tuning models.

Story map for LLMs

As we usher in the month of June, we embrace the ever-changing landscape of technological innovation, and focus on a central theme:LLM in Production.

June 2023

Forward engineering June 2023

As we look back on the past month, we're excited to share our latest engineering report, which highlights our recent achievements, projects, and insights in May 2023.

May 2023

Forward engineering May 2023

This article explores Langchain document loaders, explaining their role in overcoming token limits, integrating with vector databases, and demonstrating how to build a question-answering system using CSV data and an AI agent.

Working with langchain document loaders

This article explores strategies to overcome OpenAI's token limit using Langchain, including stuffing, map-reduce, refining, and map-rerank methods. It discusses the advantages and drawbacks of each approach in handling large documents or prompts while working within the constraints of OpenAI's API.

Workaround with OpenAI's token limit with Langchain

As we look back on the past month, we're excited to share our latest engineering report, which highlights our recent achievements, projects, and insight in March 2023.

March 2023

Forward engineering March 2023

Successfully went through 2022, a year that we might say -  a new record team reached highest growth record, kickstarted community engagement, home, compliance training & individual development are speeding up, and a whole new place to call Dwarves.

2022 in review

Forward engineering December 2023

We want to demonstrate and encourage individual and organizational learning, where both gaining and sharing knowledge is prioritized, valued, and rewarded. Learning is an ongoing process that never ends in our fast-moving industry.

2022

Forward engineering 2022

Having a blast company trip after 2 years of COVID, it didn’t happen in a common way. We planned our trip differently than others and designed apps for everyone to play as a way to bond as company outside physical activities.

Summit 2022: Engineering a good time

Our tech radar helps us assess and adopt new technologies, inspired by ThoughtWorks. Learn about its structure and how we use it.

Radar

Tech radar

We helped Open Fabric build a payment platform that makes it easier for stores to accept digital payments across Southeast Asia. Our team built the core technology and delivered a working product within a year.

Open Fabric

Building a payment platform from scratch

State, explain, link is a basic style for organizing and explaining ideas in an understandable way. This technique may have many names, but in essence it is a way to organize paragraphs in roughly 3 sentences or more.

monotykamary

Recently published

By categories

#playbook

#earn

#brainery

#AI

#forward

#build-log

#prompt

#LLM

#Market report

#data

#WALA

#opensource

#guides

#culture

#OGIF

#community

#labs-digest

#consulting

#digest

#archived

#engineering

#research

#golang

#case-study

#writing