You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
On distributed systems broadly defined and other curiosities. The opinions on this site are my own. I talked about the importance of reading foundational papers last week. To followup, here is my compilation of foundational papers in the distributed systems area. (I focused on the core distributed systems area, and did not cover networking, security, distributed ledgers, verification work etc. I e
CACM Web Account Membership in ACM includes a subscription to Communications of the ACM (CACM), the computing industry's most trusted source for staying connected to the world of advanced computing. Sign In Sign Up FoundationDB is an open-source transactional key-value store created more than 10 years ago. It is one of the first systems to combine the flexibility and scalability of NoSQL architect
Fed up with the complexity of distributed filesystems? minikeyvalue is a ~1000 line distributed key value store, with support for replication, multiple machines, and multiple drives per machine. Optimized for values between 1MB and 1GB. Inspired by SeaweedFS, but simple. Should scale to billions of files and petabytes of data. Used in production at comma.ai. A key part of minikeyvalue's simplicity
Introduction Apple is sharing changes to iOS, Safari, and the App Store, impacting developers’ apps in the European Union (EU) to comply with the Digital Markets Act (DMA). These changes create new options for developers, including how they can distribute apps on iOS, process payments, use web browser engines in iOS apps, request interoperability with iPhone and iOS hardware and software features,
From the beginning of this year, I started to take lecture courses of undergrad distributed systems course at UC Santa Cruz (CSE 138) by Lindsey Kuper. It consists of 23 lectures (you can see the schedule of topics from here) and recently I’ve finished all of them. I’m not a student at UCSC but due to the COVID-19 situation, all these lectures were delivered online and are available on YouTube. So
releasev0.10.0 has been released! Check the release notesApache Pinot™Realtime distributed OLAP datastore, designed to answer OLAP queries with low latencyUSE-CASESUser-facingData ProductsBusinessIntelligenceAnomalyDetectionSOURCESEVENTSSmart IndexBlazing-FastPerformantAggregationPre-MaterializationSegment Optimizer
2024/02/07A Distributed Systems Reading ListThis document contains various resources and quick definition of a lot of background information behind distributed systems. It is not complete, even though it is kinda sorta detailed. I had written it some time in 2019 when coworkers at the time had asked for a list of references, and I put together what I thought was a decent overview of the basics of
Open Source, Distributed, Scalable, Lightning Fast The graph database built for super large-scale graphs with milliseconds of latency. Get Started Live Demo View on Github Community Meeting Is ComingThe Nebula Graph community meeting aims to discuss the lastest progress of Nebula Graph and other topics that community members care. It will happen every two weeks online. Welcome to join us! Register
Learn about the background, challenges, and future of Airbnb’s distributed scheduling and queueing system. IntroductionAsynchronous background jobs can often dramatically improve the scalability of web applications by moving time-consuming, resource-intensive tasks to the background. These tasks are often prone to failures, and retrying mechanisms often make it even more expensive to operate appli
Endo is a JavaScript platform under development for secure communication among objects within one process and distributed between mutually suspicious machines. The foundation of Endo is Hardened JavaScript as implemented by ses, a tamper-proof JavaScript environment that allows safe execution of arbitrary programs in Compartments. Most JavaScript libraries built for Node.js, either in CommonJS or
I Migrated from a Postgres Cluster to Distributed SQLite with LiteFSNovember 21st, 2022 — 22 min read Over the last few months, I've been heads-down on building the content for EpicWeb.dev. And I've been building it all in the open on my YouTube channel. If you've been watching my EpicWeb.dev live streams, you'll know that I've been building the app we'll use to learn web dev: Rocket Rental 🚀. Wi
Read it now on the O’Reilly learning platform with a 10-day free trial. O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers. Many companies, from startups to Fortune 500 companies alike, use Node.js to build performant backend services. And engineers love Node.js for its approachable API and familiar syntax.
Encore – Backend Development Platform Encore provides a purpose-built workflow to help you create event-driven and distributed systems — from local development to your cloud on AWS & GCP. It consists of a Microservice Framework & Infrastructure SDK, a Local Development Environment with tools like tracing, and a Cloud Platform for automating CI/CD and cloud infrastructure provisioning. 🏁 Try Encor
As part of bringing myself up-to-speed after joining TigerBeetle, I wanted some background on how distributed consensus and replicated state machines protocols work. TigerBeetle uses Viewstamped Replication. But I wanted to understand all popular protocols and I decided to start with Raft. We'll implement two key components of Raft in this post (leader election and log replication). Around 1k line
Our post 6 Signs You Might be Misunderstanding ACID Transactions in Distributed Databases describes the key challenges involved in building high performance distributed transactions. Multiple open source ACID-compliant distributed databases have started building such transactions by taking inspiration from research papers published by Google. In this post, we dive deeper into Percolator and Spanne
この記事は新野淳一氏のブログ「Publickey」に掲載された「Google Cloud、分散クラウドを実現する「Google Distributed Cloud Edge」正式リリース。いわばGoogle Cloud版のAWS Outposts」(2022年4月8日掲載)を、ITmedia NEWS編集部で一部編集し、転載したものです。 米Googleは、エッジでワークロードを実行可能にするハードウェアとソフトウェアのフルマネージドサービス「Google Distributed Cloud Edge」の正式リリースを発表しました。 これはいわば、Google Cloud版の「AWS Outpost」とも言うべきもので、Google Cloudがアプライアンスやラックを提供し、それをユーザー自身のデータセンターや通信キャリア、Googleのネットワークエッジに配置。そこでワークロードを処理
Use this extension to add a highly scalable counter service to your app. This is ideal for applications that count viral actions or any very high-velocity action such as views, likes, or shares. Since Cloud Firestore has a limit of one sustained write per second, per document, this extension instead shards your writes across documents in a _counter_shards_ subcollection. Each client only increment
Microsoftは2019年10月16日(米国時間)、「Distributed Application Runtime」(Dapr)を発表した。Daprは、クラウドとエッジで動作する、耐障害性のあるステートレス、またはステートフルなマイクロサービスアプリケーションの構築を容易にする、移植可能でイベント駆動型のオープンソースサーバレスランタイム。Daprの開発は現在、α版の段階にある。 MicrosoftはDaprにより、新規にコードを作成するか、既存コードを移行するかにかかわらず、全ての開発者が任意の言語とフレームワークを使って、移植可能なマイクロサービスアプリケーションを簡単に構築できるようにすることを目指している。 Daprは、マイクロサービスアプリケーション構築のベストプラクティスを、一連のオープンなビルディングブロックとして体系化したものだ。これらのビルディングブロックにより、自
『Distributed Services with Go』は分散システム実装の入門書です。JSON/HTTP APIエンドポイントの作成から始まり、APIエンドポイントのprotobuf/gRPC化やTLS対応、そして非集中型サービスディスカバリ・オーケストレーションの実装であるSerfや分散合意アルゴリズムRaftなどを扱っています。『データ指向アプリケーションデザイン』で原理はなんとなくわかったけど分散システムを具体的にどう実装したら良いかわからないという人を対象にしているらしいです。(自分はこの書籍はまだほとんど読んでないです) この書籍では全てGoで実装していますが、理解度を確認しながらやっていきたいのでRustで実装していきます。 1. Let's Go Kubernetesなどで実行する分散システムの実装を学ぶ。その最初の一歩としてcommit log JSON over H
ウィズコロナなのかアフターコロナなのかはよくわからないけど、今回の新型コロナ感染拡大の事態を踏まえて分散アジャイルについての勘所について整理しようと思って、いろいろな調べてみた。世の中にはTips集と精神論ばかりが溢れている気がする。知りたいのは戦術だ。先人たちの知見から学ぶことはあるのだろうか? Agile Software Development with Distributed Teams: Staying Agile in a Global World 以前に「A Practical Guide to Distributed Scrum」という本を読んでいる(DAD本で紹介されていた)。 agnozingdays.hatenablog.com この本は基本的に「複数のグローバルなサイトに分散されたチームでのスクラム」について書かれていたと記憶している(若干自信はない)。今回読みたい
Unparalleled coordination and collaboration, at scaleHolochain gives digital agency to organizations, markets, and people. By combining cryptography, distributed databases, and consensus-free validation—new social magic is possible. About
As a consulting architect at Red Hat, I've had the privilege of working on legions of customer projects. Every customer brings their own challenges but I've found some commonalities. One thing most customers want to know is how to coordinate writes to more than one system of record. Answering this question typically involves a long explanation of dual writes, distributed transactions, modern alter
How my distributed team communicates so no context is left behind My team at CircleCI is made up of seven engineers, a product manager, a designer, and an engineering manager, and we are distributed all across the world: in the US from West Coast to East Coast, then to Serbia, Thailand, and finally Japan. Usually, we’re based in places like San Francisco, Chandler, Tempe, Boulder, Knoxville, Novi
No more DSLs: Implement and deploy a distributed system with a single program If you want to write a distributed system, then instead of writing a thousand different programs and configuration files in many different DSLs, you can use an approach I call "single-program systems", and write just one program. In this article, I describe how we do this at $DAYJOB, and I show some example programs. The
A distributed document database is a NoSQL database that stores semi-structured data, usually in a format similar to JSON, and is horizontally scalable across multiple machines. Document databases are very convenient for modern application development, because modern application frameworks and languages are also based on semi-structured objects instead of tabular data. They are often a better fit
Scribe: Transporting petabytes per hour via a distributed, buffered queueing system Our hardware infrastructure comprises millions of machines, all of which generate logs that we need to process, store, and serve. The total size of these logs is several petabytes every hour. The outputs are generally processed somewhere other than where they were generated: They can be relevant to a variety of dow
This post introduces the concept of durable execution, which is used by Stripe, Netflix, Coinbase, HashiCorp, and many others to solve a wide range of problems in distributed systems. Then it shows how simple it is to write durable code using Temporal’s TypeScript/JavaScript SDK. For an updated version of this post, see durable-execution.pdf Distributed systems When building a request-response mon
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く