Back to rankings

dtinit/data-transfer-project

Javadtinit.org/docs/dtp-what-is-it

The Data Transfer Project makes it easy for platforms to build interoperable user data portability features. We are establishing a common framework, including data models and protocols, to enable direct transfer of data both into and out of participating online service providers.

datatransferportabilitydata-portability
Star Growth
Stars
3.6k
Forks
495
Weekly Growth
Issues
18
1k2k3k
Jan 2018Nov 2020Sep 2023Jul 2026
ArtifactsMavengit clone https://github.com/dtinit/data-transfer-project.git
README

Data Transfer Project

Overview

The Data Transfer Project makes it easy for people to transfer their data between online services. We provide a common framework and ecosystem to accept contributions from service providers to enable seamless transfer of data into and out of their service.

Who We Are

Data Transfer Project (DTP) is a collaboration of organizations committed to building a common framework with open-source code that can connect any two online service providers, enabling a seamless, direct transfer of data.

We want all individuals across the web to be in control of their data.

More info

Current State

Data Transfer Project is in its early stages, and we are actively looking for partner organizations and individuals to contribute to the project. We are continuing to define the architecture and implementation. Since the code is in active development, please do thorough testing and verification before implementing.

Contact Info

Please contact dtp-discuss@googlegroups.com with any questions or comments.

About Us

The Data Transfer Project was formed in 2017 to create an open-source, service-to-service portability platform so that all individuals across the web could easily move their data between online service providers.

The partners in the Data Transfer Project believe portability and interoperability are central to innovation. Making it easier for individuals to choose among services facilitates competition, empowers individuals to try new services and enables them to choose the offering that best suits their needs.

We anticipate the Data Transfer Project solution will make a particularly big impact in global markets where downloading or uploading data is expensive and/or slow. The Data Transfer Project eliminates the need to download data at all. Instead, data is transferred directly between service providers.

DTP is early-stage open source code that is built and maintained entirely by DTP community members.

Related repositories
D4Vinci/Scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

PythonPyPIBSD 3-Clause "New" or "Revised" Licensecrawlercrawling
scrapling.readthedocs.io/en/latest/
70.6k7k
Asabeneh/30-Days-Of-Python

The 30 Days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days. This challenge may take more than 100 days. Follow your own pace. These videos may help too: https://www.youtube.com/channel/UC7PNRuno1rzYPb1xLa4yktw

PythonPyPI30-days-of-pythonpython
68.8k12.8k
run-llama/llama_index

LlamaIndex is the leading document agent and OCR platform

PythonPyPIMIT Licenseagentsapplication
developers.llamaindex.ai
51k7.8k
TanStack/query

🤖 Powerful asynchronous state management, server-state utilities and data fetching for the web. TS/JS, React Query, Solid Query, Svelte Query and Vue Query.

TypeScriptnpmMIT Licensereacthooks
tanstack.com/query
50k4k
metabase/metabase

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

ClojureOtheranalyticsbusinessintelligence
metabase.com
48.3k6.7k
DataExpert-io/data-engineer-handbook

This is a repo with links to everything you'd ever want to learn about data engineering

Jupyter Notebookapachesparkawesome
42.3k7.9k
SheetJS/sheetjs

📗 SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs

Apache License 2.0xlsxexcel
sheetjs.com
36.3k7.9k
vercel/swr

React Hooks for Data Fetching

TypeScriptnpmMIT Licensereacthook
swr.vercel.app
32.4k1.4k
mendableai/firecrawl

🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API.

TypeScriptnpmGNU Affero General Public License v3.0aicrawler
firecrawl.dev
29.5k2.5k
sinaptik-ai/pandas-ai

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

PythonPyPIOtherllmpandas
pandas-ai.com
23.7k2.3k
PrefectHQ/prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

PythonPyPIApache License 2.0pythonworkflow
prefect.io
23.5k2.4k
airbytehq/airbyte

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

PythonPyPIOtherdatapipeline
airbyte.com
21.7k5.3k