Temporal Python 1.0.0 – A Durable, Distributed Asyncio Event Loop
We are excited about the GA release of the Temporal Python SDK. Python is now a fully-supported workflow language in Temporal, and our use of native asyncio constructs makes it a perfect fit for Python developers looking to write durable workflows. Here are some links to read more about Temporal Python: Repository Application Development Guide Code Samples API Documentation Like essentially all of
GitHub - microsoft/retina: eBPF distributed networking observability tool for Kubernetes
Retina is a cloud-agnostic, open-source Kubernetes network observability platform that provides a centralized hub for monitoring application health, network health, and security. It provides actionable insights to cluster network administrators, cluster security administrators, and DevOps engineers navigating DevOps, SecOps, and compliance use cases. Retina collects customizable telemetry, which c
SQLite is fast, really fast. On a single ~40€ / m commodity server, it can sustain ~168,000 reads and ~8000 writes per second, concurrently. Because it's an embedded library designed for client-side applications such as embedded systems and phone & desktop applications, SQLite databases have to be co-located on your
GitHub - hatchet-dev/hatchet: A distributed, fault-tolerant task queue
Hatchet replaces difficult to manage legacy queues or pub/sub systems so you can design durable workloads that recover from failure and solve for problems like concurrency, fairness, and rate limiting. Instead of managing your own task queue or pub/sub system, you can use Hatchet to distribute your functions between a set of workers with minimal configuration or infrastructure: What Makes Hatchet
オラクル、シャーディングを利用したグローバルな分散DBの新サービス「Oracle Globally Distributed Autonomous Database」正式サービス開始
オラクル、シャーディングを利用したグローバルな分散DBの新サービス「Oracle Globally Distributed Autonomous Database」正式サービス開始 オラクルはデータベースを複数のシャードに分割し、それぞれをグローバルに分散させつつ全体を1つの論理的なデータベースとして扱えるOracle Cloudの新サービス「Oracle Globally Distributed Autonomous Database」を正式サービスとして提供開始しました。 Find out how you can more easily address data sovereignty requirements and achieve the highest possible scale and availability with @Oracle Globally Distribute
GitHub - Eventual-Inc/Daft: Distributed DataFrame for Python designed for the cloud, powered by Rust
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - TimeSurgeLabs/athenadb: 🦉⚡️Serverless, distributed vector database as an API
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
2024/02/07A Distributed Systems Reading ListThis document contains various resources and quick definition of a lot of background information behind distributed systems. It is not complete, even though it is kinda sorta detailed. I had written it some time in 2019 when coworkers at the time had asked for a list of references, and I put together what I thought was a decent overview of the basics of
Update on apps distributed in the European Union - Support - Apple Developer
Introduction Apple is sharing changes to iOS, Safari, and the App Store, impacting developers’ apps in the European Union (EU) to comply with the Digital Markets Act (DMA). These changes create new options for developers, including how they can distribute apps on iOS, process payments, use web browser engines in iOS apps, request interoperability with iPhone and iOS hardware and software features,
AWS Step Functions の Distributed Map と再実行機能を使用した効率的な ETL パイプラインの構築 | Amazon Web Services
Amazon Web Services ブログ AWS Step Functions の Distributed Map と再実行機能を使用した効率的な ETL パイプラインの構築 AWS Step Functions は、完全マネージドのビジュアルワークフローサービスで、AWS Glue、Amazon EMR、Amazon Redshift などのさまざまな抽出・変換・読み込み (Extract, Transform, Load; ETL) テクノロジーを含む複雑なデータ処理パイプラインを構築できます。個々のデータパイプラインタスクを繋ぎ、ペイロード、リトライ、エラー処理を最小限のコードで構成することで、ワークフローを視覚的に構築できます。 Step Functions は、データパイプライン内のタスクが一時的なエラーで失敗した場合、自動リトライとエラー処理をサポートしていますが、アクセ
Firefoxにプライバシーを保護しつつブラウザ使用状況を集めるシステム「Oblivious HTTP(OHTTP)」と「Distributed Aggregation Protocol(DAP)」が導入される予定
Mozillaがプライバシーを保護しつつブラウザ使用状況を収集するためにプライバシー保護技術「Oblivious HTTP(OHTTP)」と「Distributed Aggregation Protocol(DAP)」をFirefoxに導入することを発表しました。同時に、OHTTPとDAPを実装するためにテクノロジー企業「Fastly」やInternet Security Research Group(ISRG)傘下の「Divvi Up」と協力することも発表されています。 Built for Privacy: Partnering to Deploy Oblivious HTTP and Prio in Firefox https://blog.mozilla.org/en/products/firefox/partnership-ohttp-prio/ Firefox and Fastl
Retrieval Augmented Generation at scale — Building a distributed system for synchronizing and…
Disclaimer: We will go into some technical and architectural details of how we do this at Neum AI — A data platform for embeddings management, optimization, and synchronization at large scale, essentially helping with large-scale RAG. As we’ve shared in other blogs in the past, getting a Retrieval Augmented Generation (RAG) application started is pretty straightforward. The problem comes when tryi
新しく公開された Distributed Load Testing on AWS (DLT) ワークショップを試した - kakakakakku blog
負荷テストを実行したいけど，ラップトップや Amazon EC2 インスタンス1台から実行すると負荷テストを実行する側がボトルネックになってしまって，期待した負荷テストにならないという悩みはよくあると思う🔥 そこで負荷テスト専用の SaaS などを活用して負荷テストを実現する案もあるけど，AWS では「AWS ソリューションライブラリ（サービスではない）」として負荷テストを実行・管理する「Distributed Load Testing on AWS (DLT)」が提供されている❗️ aws.amazon.com 僕自身は2年ほど前に Distributed Load Testing on AWS (DLT) を検証したことがあるけど，最近 Distributed Load Testing on AWS (DLT) を体験するワークショップ（日本語🇯🇵）が公開されたらしく，復習も兼ね
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Load test your applications in a CI/CD pipeline using CDK pipelines and AWS Distributed Load Testing Solution | Amazon Web Services
AWS DevOps Blog Load test your applications in a CI/CD pipeline using CDK pipelines and AWS Distributed Load Testing Solution Load testing is a foundational pillar of building resilient applications. Today, load testing practices across many organizations are often based on desktop tools, where someone must manually run the performance tests and validate the results before a software release can b
The BBC on Mastodon: experimenting with distributed and decentralised social media
The BBC on Mastodon: experimenting with distributed and decentralised social media Posted by Tristan Ferne on 31 Jul 2023, last updated 31 Jul 2023 As the social media landscape ebbs and flows, the team at BBC Research & Development are researching social technologies and exploring possibilities for the BBC. One part of our work is to establish a BBC presence in the distributed collection of socia
LiteFS Cloud: Distributed SQLite with Managed Backups Author Name Darla Magdalene Shockley Author Name Ben Johnson @benbjohnson @benbjohnson Image by Annie Ruygt With Fly.io, you can get your app running globally in a matter of minutes, and with LiteFS, you can run SQLite alongside your app! Now we’re introducing LiteFS Cloud: managed backups and point-in-time restores for LiteFS—whether your app
FoundationDB: A Distributed Key-Value Store – Communications of the ACM
CACM Web Account Membership in ACM includes a subscription to Communications of the ACM (CACM), the computing industry's most trusted source for staying connected to the world of advanced computing. Sign In Sign Up FoundationDB is an open-source transactional key-value store created more than 10 years ago. It is one of the first systems to combine the flexibility and scalability of NoSQL architect
[re:Inforce] 組織によるセキュリティレベルの向上事例 | APS201 How AWS and MongoDB raise the security bar with distributed ownership セッションレポート #AWSreInforce | DevelopersIO
[re:Inforce] 組織によるセキュリティレベルの向上事例 | APS201 How AWS and MongoDB raise the security bar with distributed ownership セッションレポート #AWSreInforce AWS re:Inforce 2023のセッション、How AWS and MongoDB raise the security bar with distributed ownershipのレポートです。 こんにちは、菊池です。 AWS re:Inforce 2023参加中です。この記事は、セッション APS201 How AWS and MongoDB raise the security bar with distributed ownershipのレポートです。 セッション概要（公式より引用） In this se
![[re:Inforce] 組織によるセキュリティレベルの向上事例 | APS201 How AWS and MongoDB raise the security bar with distributed ownership セッションレポート #AWSreInforce | DevelopersIO](https://cdn-ak-scissors.b.st-hatena.com/image/square/c599b8acb6609a50aa4e503b6805c190c33b588e/height=288;version=1;width=512/https%3A%2F%2Fdevio2023-media.developers.io%2Fwp-content%2Fuploads%2F2023%2F06%2Freinforce-APS201-001.jpg)
分散トレーシングとOpenTelemetryのススメ / Getting started distributed tracing and OpenTelemetry
OCHaCafe Season7 の LT 資料です。 https://ochacafe.connpass.com/event/282817/
GitHub - ArroyoSystems/arroyo: Distributed stream processing engine in Rust
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Implementing the Raft distributed consensus protocol in Go | notes.eatonphil.com
As part of bringing myself up-to-speed after joining TigerBeetle, I wanted some background on how distributed consensus and replicated state machines protocols work. TigerBeetle uses Viewstamped Replication. But I wanted to understand all popular protocols and I decided to start with Raft. We'll implement two key components of Raft in this post (leader election and log replication). Around 1k line
Facebook LLAMA is being openly distributed via torrents | Hacker News
It seems that the leak originated from 4chan [1]. Two people in the same thread had access to the weights and verified that their hashes match [2][3] to make sure that the model isn't watermarked. However, the leaker made a mistake of adding the original download script which had his unique download URL to the torrent [4], so Meta can easily find them if they want to.[1]: https://boards.4channel.o
Introducing Service Weaver: A Framework for Writing Distributed Applications
The latest news from Google on open source releases, major projects, events, and student outreach programs. We are excited to introduce Service Weaver, an open source framework for building and deploying distributed applications. Service Weaver allows you to write your application as a modular monolith and deploy it as a set of microservices. More concretely, Service Weaver consists of two core pi
About Grafana TempoGrafana Tempo lets you scale tracing as far as possible with minimal operational cost and less complexity than ever before. The easiest way to get started is with Grafana Cloud, our fully composable observability stack. Introduction to Grafana TempoGrafana Tempo is an open source, easy-to-use, and high-scale distributed tracing backend. Tempo is cost-efficient, requiring only ob
This post introduces the concept of durable execution, which is used by Stripe, Netflix, Coinbase, HashiCorp, and many others to solve a wide range of problems in distributed systems. Then it shows how simple it is to write durable code using Temporal’s TypeScript/JavaScript SDK. For an updated version of this post, see durable-execution.pdf Distributed systems When building a request-response mon
I Migrated from a Postgres Cluster to Distributed SQLite with LiteFS
I Migrated from a Postgres Cluster to Distributed SQLite with LiteFSNovember 21st, 2022 — 22 min read Over the last few months, I've been heads-down on building the content for EpicWeb.dev. And I've been building it all in the open on my YouTube channel. If you've been watching my EpicWeb.dev live streams, you'll know that I've been building the app we'll use to learn web dev: Rocket Rental 🚀. Wi
AWS Step Functionsで大規模な並列処理を実施できるDistributed Mapを試してみた #reinvent | DevelopersIO
こんにちは、CX事業本部 IoT事業部の若槻です。 先日開催されたAWS re:Invent 2022で、AWS Step Functionsで大規模な並列処理を実施できるStep Functions Distributed Mapが発表されました。 AWS Step Functions launches large-scale parallel workflows for data processing and serverless applications Step Functions Distributed Map – A Serverless Solution for Large-Scale Parallel Data Processing | AWS News Blog S3 Bucket内のオブジェクトをもとに大規模なMap処理が行えるとのことで、実際に現地会場でKeynote
25年前のAmazonの分散コンピューティングの考え方を要約した「The Distributed Computing Manifesto」文書が公開されました #reinvent | DevelopersIO
re:Invent2022 4日目、Dr. Werner Vogelsのキーノートにおいて、「The Distributed Computing Manifesto」というドキュメントがNew Articleとして紹介されました。 The Distributed Computing Manifesto | All Things Distributed Amazonという超巨大なサービスが、モノリスの状態からその時まさに分散コンピューティングを推進していく転換点における考え方を示した貴重なドキュメントになっており、また、Werner自身も言うように、今後のAWSの進化の方向性を示唆する内容にもなっています。 温故知新ってこと…?!! ( ﾟдﾟ) ｶﾞﾀｯ / ヾ ＿_L| /￣￣￣/＿ ＼/ / まさにそんな雰囲気。 文書の位置づけ こちらのサイトに追加されたドキュメントという位置
[速報]Step Functionsで大量データの並列処理が可能に！Step Functions Distributed MapがGAされました！# reInvent | DevelopersIO
速報です！ 本日の Werner Vogels キーノートで、Step Functionsで大規模な並列処理を可能にする、「Step Functions Distributed Map」が発表されました！ これまで Step FunctionsにはもとよりMap並列処理を提供機能がありましたが、並列処理の同時実行数は最大40回という制限がありました。 Step Functions Distributed Mapで何ができるのか Mapは上記の制限により、数千件以上の項目を並列でタイムリーに処理することは困難でしたが、「Step Functions Distributed Map」により大規模な並列ワークロードの処理が可能になりました！！ 「Step Functions Distributed Map」では、最大1万回の並列ワークロードを起動することができます。並列で実行できる処理には、St
![[速報]Step Functionsで大量データの並列処理が可能に！Step Functions Distributed MapがGAされました！# reInvent | DevelopersIO](https://cdn-ak-scissors.b.st-hatena.com/image/square/0811654a8dac484dafd58339e445d07a684c2164/height=288;version=1;width=512/https%3A%2F%2Fdevio2023-media.developers.io%2Fwp-content%2Fuploads%2F2022%2F12%2Fstep-function-destributed-map-reinvent1.png)
Step Functions Distributed Map – A Serverless Solution for Large-Scale Parallel Data Processing | Amazon Web Services
AWS News Blog Step Functions Distributed Map – A Serverless Solution for Large-Scale Parallel Data Processing I am excited to announce the availability of a distributed map for AWS Step Functions. This flow extends support for orchestrating large-scale parallel workloads such as the on-demand processing of semi-structured data. Step Function’s map state executes the same processing steps for multi
Hive Distributed Profiling System in Treasure Data - Japanese version #tdtechtalk
『Hive Distributed Profiling System in Treasure Data』の日本語版スライド。 English version -> https://speakerdeck.com/okumin/hive-distributed-profiling-system-in-treasure-data-english-version-number-tdtechtalk TreasureData Tech Talk 2022にて発表 https://techplay.jp/event/879660
The Distributed Computing ManifestoNovember 16, 2022 • 3941 words Today, I am publishing the Distributed Computing Manifesto, a canonical document from the early days of Amazon that transformed the architecture of Amazon’s ecommerce platform. It highlights the challenges we were facing at the end of the 20th century, and hints at where we were headed. When it comes to the ecommerce side of Amazon,
GitHub - CarperAI/trlx: A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - tigerbeetle/tigerbeetle: The distributed financial transactions database designed for mission critical safety and performance.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Cloudflare Queues: globally distributed queues without the egress fees
Cloudflare Queues: globally distributed queues without the egress fees09/27/2022 Developers continue to build more complex applications on Cloudflare’s Developer Platform. We started with Workers, which brought compute, then introduced KV, Durable Objects, R2, and soon D1, which enabled persistence. Now, as we enable developers to build larger, more sophisticated, and more reliable applications, i
Docs Index Get Started with LiteFS Toggle Get Started with LiteFS section LiteFS overview Speedrun: Adding LiteFS to your app Getting Started with LiteFS on Fly.io Getting Started with LiteFS in Docker How LiteFS Works FAQ LiteFS Cloud Toggle LiteFS Cloud section Using LiteFS Cloud for Backups Restoring from a LiteFS Cloud Backup Disaster Recovery from LiteFS Cloud Using LiteFS Toggle Using LiteFS
A minimal distributed key-value database with Hashicorp's Raft library | notes.eatonphil.com
A minimal distributed key-value database with Hashicorp's Raft library When I wrote the "build a distributed PostgreSQL proof of concept" post I first had to figure out how to use Hashicorp's Raft implementation. There weren't any examples I could find in the Hashicorp repo itself. And the only example I could find was Philip O'Toole's hraftd. It's great! However, I have a hard time following mult
You’re seeing information for Bulgaria . To see local features and services for another location, select a different city. Show more Engineering, BackendMySQL to MyRocks Migration in Uber’s Distributed DatastoresSeptember 1, 2022 / Global Introduction Uber uses MySQL as the underlying database engine for Schemaless and Docstore, our distributed databases. By default, MySQL uses the most popular In
GitHub - maxpert/marmot: A distributed SQLite replicator built on top of NATS
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Distributed SQLite: Paradigm shift or hype?
A Distributed Systems Reading List
GitHub - runabol/tork: A distributed workflow engine
LiteFS Cloud: Distributed SQLite with Managed Backups
Grafana Tempo OSS | Distributed tracing backend
Building Reliable Distributed Systems in Node.js
The Distributed Computing Manifesto
LiteFS - Distributed SQLite
MySQL to MyRocks Migration in Uber's Distributed Datastores