You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - cumulo-autumn/StreamDiffusion: StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - langgenius/dify: Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to p
Dify Cloud · Self-hosting · Documentation · Enterprise inquiry Dify is an open-source LLM app development platform. Its intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production. Here's a list of the core features: 1. Workflow: Build and test powerful AI workflows on a visual ca
GitHub - radames/Real-Time-Latent-Consistency-Model: Demo showcasing ~real-time Latent Consistency Model pipeline with Diffusers and a MJPEG stream server
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.
【インターンレポート】LINEの大規模ETL batch pipelineにおけるdbtの導入によるデータ分析での課題解決の検証
LINE株式会社は、2023年10月1日にLINEヤフー株式会社になりました。LINEヤフー株式会社の新しいブログはこちらです。 LINEヤフー Tech Blog こんにちは、早稲田大学政治経済学経済学科 学部3年の滝田愛澄と申します。2023年8月7日から6週間、LINE株式会社のIU Data Connectチームにて、就業型インターンシップに参加させていただきました。本インターンでは、LINEの大規模ETL batch pipelineであるVinitusが現在抱えている課題を解決することを目的に、data build tool (dbt) の調査とdbtを用いた新たなworkflowのプロトタイプの設計・実装に取り組みました。このレポートでは、現在のVinitusが抱えている課題を確認し、dbtの導入によってどのようにそれらの課題を解決できるか、具体的にこのプロトタイプでは何をど
Case studyHow Brainly avoids workflow bottlenecks with automated tracking Case studyHow Neptune gave Waabi organization-wide visibility on experiment data
GitHub - catchpoint/workflow-telemetry-action: Github action to collect metrics (CPU, memory, I/O, etc ...) from your workflows to help you debug and optimize your CI/CD pipeline
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
GitHub - HojiChar/HojiChar: The robust text processing pipeline framework enabling customizable, efficient, and metric-logged text preprocessing.
Text preprocessing is far from a one-size-fits-all process. Depending on the data source and the specific task at hand, various steps including normalization, noise removal, and filtering may be necessary. Not all texts require the same level of preprocessing. For instance, relatively clean texts may only need minimal filtering, while "dirtier" sources like Common Crawl data often require more tho
GitHub - diggerhq/digger: Digger is an open source IaC orchestration tool. Digger allows you to run IaC in your existing CI pipeline ⚡️
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
We introduce StreamDiffusion, a real-time diffusion pipeline designed for interactive image generation. Existing diffusion models are adept at creating images from text or image prompts, yet they often fall short in real-time interaction. This limitation becomes particularly evident in scenarios involving continuous input, such as Metaverse, live video streaming, and broadcasting, where high throu
CloudFormationがCD Pipelineなしで そのままGit連携デプロイが可能に！ #AWSreInvent | DevelopersIO
CloudFormationがCD Pipelineなしで そのままGit連携デプロイが可能に！ #AWSreInvent ども、re:Invent現地参加組の もこ@札幌オフィス です。 まだ現地ではキーノートが始まっておらず、まだラスベガスにも到着出来ていない現状ですが、ぽろぽろと出てきているアップデートをご紹介していきます！ CloudFormationのStack更新がGitからできるようになった！ このサービス、簡潔にまとめると「CloudFormationのStack deployment fileを元にして良い感じに指定したブランチに更新があったらデプロイするよ！」といったサービスです。 Stack deployment fileを簡単に説明するとCloudFormationのTemplateにおけるパラメータを定義できるモノです。 今回のアップデートであるGitとの同期機能
Best practices for managing Terraform State files in AWS CI/CD Pipeline | Amazon Web Services
AWS DevOps Blog Best practices for managing Terraform State files in AWS CI/CD Pipeline Introduction Today customers want to reduce manual operations for deploying and maintaining their infrastructure. The recommended method to deploy and manage infrastructure on AWS is to follow Infrastructure-As-Code (IaC) model using tools like AWS CloudFormation, AWS Cloud Development Kit (AWS CDK) or Terrafor
はじめに PyAirbyteがリリースされました。(2024/03/16時点ではBeta版なのでご注意を) PyAirbyteはExtractのコネクタ部分をPythonのライブラリとして提供してPandasに格納するという機能を提供しているらしい。 つまり、BigQueryのクライアントと合わせればExtractとLoadの部分を過疎結合にしつつ、スケジューラーでPythonを呼び出すだけのシンプルなData Injest Pipelineを作ることが可能なのでは！？ということで検証します。 個人的に考えるData Injestツールの抱える課題点 FivetranのようなSaaSを使い始める際は規約確認や、契約がとても面倒 Airbyteは契約関連の面倒な部分は無いが、運用工数が大きすぎる worker, sever, temporal, api, dbなどなど(ちゃんと拡張性を考えて
CI/CDパイプラインをコード化することで継続的な構築・メンテナンスを可能にするPipeline as Codeという取り組みが注目を集めている。システム開発で当たり前になったコードのバージョン管理や自動テスト・実行環境のメンテナンスがCI/CDパイプラインにも及んでいるのだ。 本セッションでは、NTTコミュニケーションズの杉野博徳氏が、自分たちのチームで取り組んできたPipeline as Codeについて解説した。杉野氏は、NTTコミュニケーションズの社内向けSREという立場で、3年にわたって自分たちと複数のアプリケーション開発チームのCI/CDパイプラインを構築・メンテナンスしてきた。開発チームなどで継続的デリバリーの環境構築・メンテナンスに取り組むエンジニアにとって役立つ事例となるだろう。 CI/CDパイプラインで考慮すべき3つのポイント NTTコミュニケーションズ株式会社 イノベ
GitHub - pchunduri6/rag-demystified: An LLM-powered advanced RAG pipeline built from scratch
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
A complete solution for engineering teams of all sizes.Pull requestsCompare revisionsBrowse codeQuality GatesWebhooksProtected branchesNotifications and alerts
Rebuilding Netflix Video Processing Pipeline with Microservices
Liwei Guo, Anush Moorthy, Li-Heng Chen, Vinicius Carvalho, Aditya Mavlankar, Agata Opalach, Adithya Prakash, Kyle Swanson, Jessica Tweneboah, Subbu Venkatrav, Lishan Zhu This is the first blog in a multi-part series on how Netflix rebuilt its video processing pipeline with microservices, so we can maintain our rapid pace of innovation and continuously improve the system for member streaming and st
Load test your applications in a CI/CD pipeline using CDK pipelines and AWS Distributed Load Testing Solution | Amazon Web Services
AWS DevOps Blog Load test your applications in a CI/CD pipeline using CDK pipelines and AWS Distributed Load Testing Solution Load testing is a foundational pillar of building resilient applications. Today, load testing practices across many organizations are often based on desktop tools, where someone must manually run the performance tests and validate the results before a software release can b
GitHub - srevinsaju/togomak: A declarative pipeline orchestrator with the magic of HCL as a configuration language, inspired from Terraform's architecture.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
ML Pipeline Architecture Design Patterns (With Examples)
StreamDiffusion: A Pipeline-level Solution for Real-time Interactive Generation
PyAirbyteで始める簡単Data Injest Pipeline
CI/CDパイプラインをコード化する「Pipeline as Code」とは？ 構築・メンテナンスを実現する5ステップ
Gitness: Open-Source Code Hosting & CI/CD Pipeline Engine