サクサク読めて、アプリ限定の機能も多数!
トップへ戻る
猫
yu-ishikawa.medium.com
We are all seeing the rapid growth of dbt’s popularity today. The number of users and the size of its community is on the rise. There are various aspects of dbt which attract experts in the data world. I love all of the features of dbt too. Specifically, I love dbt packages, because it makes it easy to extend dbt’s functionality. Moreover, we can reuse dbt packages which others developed, finding
Today, we are bringing many DevOps concepts to the data engineering world. One of the most basic measures is static code analysis used to flag programming errors, bugs, stylistic errors and suspicious constructs. It makes our programming code more readable and maintainable as well. Especially, integrating static code analysis tools with continuous integration (CI) is very useful. As I am a dbt lov
How do you implement and test data pipelines with BigQuery to create intermediate tables and manage metadata and data discovery? I used to use Apache Airflow’s operators with BigQuery. However, I basically need to implement code in python and manage the dependencies between BigQuery tables manually. As well as, actually, apache airflow enables us to test BigQuery tables with the CheckOperator . Bu
I have been addressing how I should maintain data quality over years as a machine learning enginner. I think checking data quality has been bothering all data guys as well. We had to inspect whether there is something wrong with data. What if can we make that automatic? Apache Airflow has a huge possibility to make our dream come true. In this article, I would like to make airflow a automatic data
I described how to serve trained tensorflow models with tensorflow serving in Serving Pre-Modeled and Custom Tensorflow Estimator with Tensorflow Serving before. In the article, I explained how to make tensorflow models with estimator and how to serve the models with tensorflow serving and docker. And tensorflow serving starts supporting the RESTful API feature at the version 1.8 in addition to gR
Databricks announced a brand new end-to-end machine learning platform which is an open source project. Nowadays, end-to-end machine learning platform is getting important. We data science guys want to focus on building models as much as possible. But traditionally, we need a bunch of cost to build teams for the end-to-end pipelines. Related WorksRecently, there are several machine learning platfor
2017–12–13 JST になりました.mixi の OB/OG の方たちによる ex-mixi Advent Calendar 2017 の13日目を担当します.この記事では,日本からサンフランシスコに引っ越すことになったときに,どういうことを知っておくと良いかを解説します.あくまでも筆者個人の経験を通して得たものなので,すべてがこの通りにすればよいと言うものではありません.また筆者が移住したのは 2016 年なので,制度や仕組みが変わったりしている可能性もあるのでご留意ください.しかし,どなたかの参考になれが幸いです. 自己紹介mixi には,2010年から2013年まで在籍していました.たんぽぽグループ解析チームの一員として,Hadoop/Hiveなどを利用した大規模データ解析基盤の構築やデータ分析などデータなにがし全般を担当してました.今や日本のウェブ業界においてもデータサイエ
このページを最初にブックマークしてみませんか?
『yu-ishikawa.medium.com』の新着エントリーを見る
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く