GitHub - xslogic/phoebus: Phoebus is a distributed framework for large scale graph processing written in Erlang.
* UPDATE[29/11/2010] : Added support for 'Aggregate' function PHOEBUS ======= Contents -------- * Introduction * Computational Model * Distributed Processing * Getting it to work * Pluggable Storage and True Multi-Node Distribution with HDFS and Thrift * Next Steps * References Introduction ------------ Phoebus is a system written in Erlang for Distributed processing of very large graphs that span
SE-Radio Episode 285: James Cowling on Dropbox’s Distributed Storage System : Software Engineering Radio
James Cowling of Dropbox (architect of their distributed storage system), speaks to Robert Blumen about their move from Amazon’s S3 to their own infrastructure; The show covers: the size, scope and scale of Dropbox’s data management; their experience on Amazon’s S3; why S3 over time did not meet their needs; how the decision was made to build their own; the architecture of Magic Pocket; building t
コマンド リポジトリの作成 すでにあるファイルからリポジトリを作成するには init コマンドと add コマンドを用いる。 % hg init % hg add -X *.bak % hg commit -m 'Initial revision.' 既存の repo をコピーするには clone を用いる。 この場合は .hg/hgrc の paths セクションに default URL が設定される。 % hg clone ssh://hgserv.domain/projects/subproject 情報収集 hg status # 変更されたファイル hg diff # ファイルに対する変更 hg log # 変更履歴 hg log FILE ... SUBDIR/ # ファイル・ディレクトリの変更履歴 hg annotate FILE # 誰がいつ何を変更したか？ hg tip
GitHub - Netflix/EVCache: A distributed in-memory data store for the cloud
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Jeff Dean: Design Lessons and Advice from Building Large Scale Distributed Systems – Perspectives
Disclaimer: The opinions expressed here are my own and do not necessarily represent those of current or past employers. Recent CommentsMatt on Seagate HAMRJames Hamilton on Seagate HAMRMatt on Seagate HAMRJames Hamilton on Cost of Power in Large-Scale Data CentersChris K. on Cost of Power in Large-Scale Data CentersJames Hamilton on Cost of Power in Large-Scale Data CentersRob F on Cost of Power i
Distributed TensorFlow を Google Cloud Platform で動かしてみる - Qiita
さて前々回、前回とDistributed Tensorflowの仕組みを学んできましたが、いよいよGoogle Cloud Platform(GCP)上で動かしてみます。無料枠の\$300を突っ込んでvCPUインスタンスをたくさん作れば貧乏人にも高火力の鱗片が見れるのでは？という期待がモチベーションでした。 最初にネタばらししてしまうと、GCPの無料アカウントでは1regionにつきvCPU8個までに制限されていることがわかりましたので、それほどの高火力にはなりませんでした。今回はインターフェイスとマスタサーバに１個ずつvCPUを当てたので、vCPU6個での並列化となります。それでも、もちろんちゃんと速くなりました。 Dockerイメージの作成 GCPのContainer Engine で動かしたいので、まずDockerイメージを作ります。 先日ビルドしたTensorflowサーバのみを入
Parapet - A purely functional library to build distributed and event-driven systems
A purely functional library to build distributed and event-driven systems It's not a secret that writing distributed systems is a challenging task that can be logically broken into two main aspects: implementing distributed algorithms and running them. Parapet plays the role of execution framework for distributed algorithms - it can be viewed as an intermediate layer between a low-level effect lib
GitHub - apache/gobblin: A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Mio - a distributed Skip Graph based orderd KVS
Mio is... a distributed orderd KVS memcached + range search Skip Graph based Written in Erlang http://github.com/higepon/mioRead less
This document summarizes Cassandra, an open source distributed database management system designed to handle large amounts of data across many commodity servers. It discusses Cassandra's history, key features like tunable consistency levels and support for structured and indexed columns. Case studies describe how companies like Digg, Twitter, Facebook and Mahalo use Cassandra to handle terabytes o
December 12, 2016 Volume 14, issue 5 PDF Life Beyond Distributed Transactions An apostate's opinion Pat Helland This is an updated and abbreviated version of a paper by the same name first published in CIDR (Conference on Innovative Database Research) 2007. Transactions are amazingly powerful mechanisms, and I've spent the majority of my almost 40-year career working on them. In 1982, I first work
Avout brings Clojure's in-memory model of state to distributed application development by providing a distributed implementation of Clojure's Multiversion Concurrency Control (MVCC) STM along with distributable, durable, and extendable versions of Clojure's Atom and Ref concurrency primitives. Download on GitHub The Avout equivalent of Hello World. (use 'avout.core) (def client (connect "127.0.0.1
asyncoro: Asynchronous, Concurrent, Distributed Programming with Python — asyncoro 4.5.3 documentation
» asyncoro: Asynchronous, Concurrent, Distributed Programming with Python asyncoro: Asynchronous, Concurrent, Distributed Programming with Python¶ Note asyncoro has been renamed pycos to better reflect its functionality and to avoid confusion with asyncore module. asyncoro is a Python framework for asynchronous, concurrent, network, distributed programming and distributed computing, using generato
NSQ is a realtime message processing system designed to operate at bitly’s scale, handling billions of messages per day. It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high availability coupled with a reliable message delivery guarantee. Operationally, NSQ is easy to configure and deploy (all parameters are specified on the comma
The Performance of Distributed Data-Structures Running on a "Cache-Coherent" In-Memory Data Grid - High Scalability -
« Sponsored Post: ROBLOX, Percona, Palantir, ElasticHosts, Atlantic.Net, ScaleOut, ground(ctrl), New Relic, NetDNA, GigaSpaces, AiCache, Logic Monitor, AppDynamics, CloudSigma, ManageEngine, Site24x7 | Main | Stuff The Internet Says On Scalability For August 17, 2012 » This is a guest post by Ron Pressler, the founder and CEO of Parallel Universe, a Y Combinator company building advanced middlewar
GitHub - youtube/doorman: Doorman: Global Distributed Client Side Rate Limiting.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Managing Big Data: Architectural Approaches for making batch data available online (Hadoop and Distributed Computing at Yahoo!)
YDN Hadoop and Distributed Computing at Yahoo! Managing Big Data: Architectural Approaches for making batch data available online This is the beginning of an ongoing series of blog posts on “Managing Big Data”. This series will focus on techniques that Yahoo uses to process large volumes of data, ranging from initial collection of data to the end usage of that data. Introduction Over the last seve
horovodのご紹介 ~Tensorflowの分散処理(Distributed Tensorflow)を簡単に実現！！~ - Qiita
horovodのご紹介 ~Tensorflowの分散処理(Distributed Tensorflow)を簡単に実現！！~PythonDeepLearningPython3TensorFlowhorovod 現在、ニューラルネットワークのモデルは非常に巨大になっていて、１台のGPUマシンだと計算に多くの時間がかかってしまうことが多くなってきました。そこで、多くのGPUで並列処理を行い計算を高速化させる需要が増えています。Tensorflowにも分散実行させるしくみを利用して並列処理を行うことが可能です。しかし、Tensorflowで分散処理を行うには以下のことを理解して１GPU用のコードを書き換える必要があります Parmeter server、Workerの配置 各サーバーへのグラフ、データの配置 これらを意識してコードを書き換えることは可能ですが、学習コストと改造量が多くDistrib
dshとは dshは，リモートシェル(rsh/remsh/ssh)を複数同時に実行するためのラッパープログラムです． rsh/remsh/sshはシェル(ksh/bash/zsh/tcsh/sh等)をリモートホストで実行するためのプログラムです． セキュリティーホールについて Bugtraqに２００３年８月１０日に流れたSecurity advisory アドバイザリのページ はLinux等の asprintf を提供しているシステムには影響しません． つまりglibcを使っている Debian や ほとんどのLinuxは脆弱ではありません． また，dshの提供するasprintf の実装を確認したところ，vsnprintf で バッファ長を指定しているため，脆弱ではないです． クラッシュさせることはできませんでした． "make check"でセキュリティーホールが存在するかどうかを確認
Google Cloud、Google Distributed Cloudにオンプレミス用の「Anthos on-premises」を「GDC Virtual」として統合
Google Cloud、Google Distributed Cloudにオンプレミス用の「Anthos on-premises」を「GDC Virtual」として統合 Google Cloudは、これまで同社のハイブリッドクラウド戦略を実現する製品としてオンプレミス用に提供してきた「Anthos on-premises」の製品名を「Google Distributed Cloud Virtual」（GDC Virtual）に変更し、Google Distributed Cloudブランド傘下の製品として位置づけ直すことを明らかにしました。 New update to our Google Distributed Cloud portfolio Google Distributed Cloud Virtual: a software-only solution that brings o
GitHub - hyperopt/hyperopt: Distributed Asynchronous Hyperparameter Optimization in Python
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
MapReduce-0.01 - Perl version of Google's distributed data processing - metacpan.org
The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
PEP 374 – Choosing a distributed VCS for the Python project | peps.python.org
PEP 374 – Choosing a distributed VCS for the Python project Author: Brett Cannon <brett at python.org>, Stephen J. Turnbull <stephen at xemacs.org>, Alexandre Vassalotti <alexandre at peadrop.com>, Barry Warsaw <barry at python.org>, Dirkjan Ochtman <dirkjan at ochtman.nl> Status: Final Type: Process Created: 07-Nov-2008 Post-History: 07-Nov-2008, 22-Jan-2009 Table of Contents Rationale Terminolog
GitHub - yugabyte/yugabyte-db: YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.
NSQ Docs 0.2.24-alpha - a realtime distributed messaging platform
a realtime distributed messaging platform by Matt Reiferson @imsnakes and Jehiah Czebotar @jehiah Introduction NSQ is a realtime distributed messaging platform designed to operate at bitly’s scale, handling billions of messages per day (current peak of 90k+ messages per second). It promotes distributed and decentralized topologies without single points of failure, enabling fault tolerance and high
GitHub - transient-haskell/transient: A full stack, reactive architecture for general purpose programming. Algebraic and monadically composable primitives for concurrency, parallelism, event handling, transactions, multithreading, Web, and distributed com
A full stack, reactive architecture for general purpose programming. Algebraic and monadically composable primitives for concurrency, parallelism, event handling, transactions, multithreading, Web, and distributed computing with complete de-inversion of control (No callbacks, no blocking, pure state) License
GitHub - osrg/namazu: :fish: 鯰: Programmable fuzzy scheduler for testing distributed systems
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Randomly distributed embedding making short-term high-dimensional data predictable | PNAS
Randomly distributed embedding making short-term high-dimensional data predictable aSchool of Mathematical Sciences, Soochow University, Suzhou 215006, China;bInstitute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan;cSchool of Mathematical Sciences, Fudan University, Shanghai 200433, China;dCenter for Computational Systems Biology, Institute of Science and Technology for Bra
GitHub - yaledb/calvin: Calvin is a scalable transactional database system that leverages determinism to guarantee active replication and full ACID-compliance of distributed transactions without two-phase commit. Most of the code is for VLDB 2014 paper: 《
Calvin is a scalable transactional database system that leverages determinism to guarantee active replication and full ACID-compliance of distributed transactions without two-phase commit. Most of the code is for VLDB 2014 paper: 《An Evaluation of the Advantages and Disadvantages of Deterministic Database Systems》.
In the world of decentralization, distributed hash tables (DHTs) recently have had a revolutionary effect. The chaotic, ad hoc topologies of the first-generation peer-to-peer architectures have been superseded by a set of topologies with emergent order, provable properties and excellent performance. Knowledge of DHT algorithms is going to be a key ingredient in future developments of distributed a
DDoS攻撃（Distributed Denial of Service Attack）とは「分散型サービス妨害攻撃」とも呼ばれるサイバー攻撃の手法の1つである。 DDoS攻撃（Distributed Denial of Service Attack）とは「分散型サービス妨害攻撃」とも呼ばれるサイバー攻撃の手法の1つである。攻撃者が複数の第三者のコンピュータに攻撃用のスクリプトやプログラムを仕掛け、一斉に大量のデータを標的の公開サービス（サーバ）に送りつけることで正常に稼働できなくする攻撃である。他にも、本当の攻撃を隠すためのダミー攻撃としても使われることがある。 Distributedではない（分散ではない）手法としてDoS攻撃があるが、これは攻撃者や第三者の1台のコンピュータから標的のサービスに対して大量のデータを送りつけてくる攻撃である。攻撃される側は、気付きやすく、遮断しやすい。
Elasticsearch: The Official Distributed Search & Analytics Engine | Elastic
Observability, security, and search solutions — powered by the Elasticsearch Platform.
GitHub - akka/akka: Build highly concurrent, distributed, and resilient message-driven applications on the JVM
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Building Distributed Systems with Netflix OSS and Spring Cloud
As presented at: http://www.meetup.com/Pivotal-Open-Source-Hub/events/219264521/ With the advent of microservice and cloud-native application architectures, building distributed systems is becoming increasingly common for the enterprise Java developer. Fortunately many of the innovators in the space, including Twitter, LinkedIn, and Netflix, have embraced the JVM as they’ve built increasingly comp
GitHub - mozilla/persona: Persona is a secure, distributed, and easy to use identification system.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
DMTCP (Distributed MultiThreaded Checkpointing) transparently checkpoints a single-host or distributed computation in user-space -- with no modifications to user code or to the O/S. It works on most Linux applications, including Python, Matlab, R, GUI desktops, MPI, etc. It is robust and widely used (on Sourceforge since 2007). Among the applications supported by DMTCP are MPI (various implementat
GitHub - thinkaurelius/titan: Distributed Graph Database
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Early this year, the Internet Archive put out a call for a distributed web. We heard them loud and clear. Today I’m making an announcement that begins our long journey to the future of the web. A web that is faster, more secure, more robust, and more permanent. Neocities has collaborated with Protocol Labs to become the first major site to implement IPFS in production. Starting today, all Neocitie
GitHub - Yelp/paasta: An open, distributed platform as a service
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - asynkron/protoactor-go: Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Mercurial - a distributed SCM tool
The Cassandra Distributed Database
Life Beyond Distributed Transactions - ACM Queue
Avout: Distributed State in Clojure
NSQ: realtime distributed message processing at scale
DSH - dancer's shell / distributed shell
Distributed Hash Tables, Part I | Linux Journal
DDoS攻撃（Distributed Denial of Service Attack）とは
DMTCP : Distributed MultiThreaded Checkpointing
HTTP is obsolete. It's time for the Distributed Web