科学数据共享系统的现状与趋势

doi:10.19788/j.issn.2096-6369.190409

摘要/Abstract

摘要：

数据密集型科研已经成为大数据时代科学发展的新范式，科学数据开放共享已成科技界的普遍共识。在长期实践中，科学数据共享形成了以科学仪器、数据平台、数据出版、众包处理、数据交易等为典型代表的不同模式。与之对应，针对不同的领域和应用场景出现了种类繁多的解决方案，如仓储型、联邦服务型、数据分发型和按需计算与分析云服务型等。本文在对上述四类主流科学数据共享系统的服务内容、技术特征、应用场景与代表性系统分析比较的基础上，提出科学数据共享系统未来发展的的趋势，并以中国科学院战略性科技先导专项“地球大数据科学工程”研发的地球大数据云服务平台为典型案例，进行了深入的剖析。本文认为，未来的科学数据共享系统将围绕着科学数据全生命周期管理的需求，形成具备数据获取、存储、分发共享、计算分析、智能服务等功能于一体的融合型云服务系统，并将实现数据的FAIR化、智能关联和机器可理解，促进数据共享良性生态的形成。

关键词: 科学数据共享系统, 数据共享, 数据融合, 智能处理, 数据生态, 科学数据管理, 科学数据, 数据系统

Abstract:

Data-intensive research is emerging as a new paradigm for science discovery in the era of big data, and the use of open data has become common in the scientific community. Over time, different models of scientific data sharing have emerged, including scientific instruments models, data platforms models, data publishing models, crowdsourcing and data market models. Correspondingly, a variety of solutions have emerged for different fields and applications, such as data repositories, data federated services systems, data distribution systems, and on-demand computing and analysis cloud services systems. This paper examines development and future trends in scientific data sharing systems, using the Big Earth Data Cloud Services Platform as an example. It analyzes and compares the typical services and technical characteristics, using scenarios and representative systems of the above-mentioned four types of mainstream scientific data sharing systems. Our analysis suggests that future scientific data sharing systems will focus on the need to manage the full life-cycle of scientific data and will converge into a cloud service system providing functions such as data acquisition, storage, distribution and sharing, analysis, and intelligent services. By making data FAIR (Findable, Accessible, Interoperable and Reusable), machine understandable and AI-Ready, promote the formation of data sharing eco-systems.

Key words: scientific data sharing system, data sharing, data fusion, intelligent processing, data ecology, scientific data management, scientific data, data system

中图分类号:

TP315

李云婷, 温亮明, 张丽丽, 黎建辉. 科学数据共享系统的现状与趋势[J]. 农业大数据学报, 2019, 1(4): 86-97.

Yunting Li, Liangming Wen, Lili Zhang, Jianhui Li. The Status and Trends of Scientific Data Sharing Systems[J]. Journal of Agricultural Big Data, 2019, 1(4): 86-97.

图/表 9

图 1

表1

图 2

表2

图 3

图 4

图 5

表3

图 6

参考文献 61

1	国家科技基础条件平台中心.国家科学数据资源发展报告(2017)[M].北京:科学技术文献出版社,2018:34.
	National Science and Technology Infrastructure Center.National Scientific Data Resource Development Report(2017)[M].Beijing:Scientific and Technology Documentation Press,2018:34.
2	黎建辉,吴超,张丽丽,等.科学数据出版调查与分析[J].中国科学数据,2016,1(1):64-74.
	Li J H,Wu C,Zhang L L,et al.Survey and Analysis of Scientific Data Publishing[J].China Scientific Data,2016,1(1):67-74.
3	Christine L,Borgman.The Conundrum of Sharing Research Data[J].Journal of the American Society for Information Science and Technology,2012,63(6):1059-1078.
4	黎建辉,李跃鹏,王华进,等.科学大数据管理技术与系统[J].中国科学院院刊,2018,33(8):796-803.
	Li J H,Li Y P,Wang H J,et al.Scientific Big Data Management Technique and System[J].Bulletin of Chinese Academy of Sciences,2018,33(8):796-803.
5	黎建辉,沈志宏,孟小峰.科学大数据管理:概念、技术与系统[J].计算机研究与发展,2017,54(2):235-247.
	Li J H,Shen Z H,Meng X F.Scientific Big Data Management:Concepts, Technologies and System[J].Journal of Computer Research and Development,2017,54(2):235-247.
6	Grossman R.Data Lakes, Clouds, and Commons: A Review of Platforms for Analyzing and Sharing Genomic Data[J].Trends in Genetics,2019,35(3):223-234.
7	张丽丽.科学数据共享治理:模式选择与情景分析[J].中国图书馆学报,2017(2):54-65.
	Zhang L L.Scientific Data Sharing Governance:Model Selection and Scenario Analysis[J].Journal of Library Science in China,2017(2):54-65.
8	李成赞,张丽丽,侯艳飞,等.科学大数据开放共享:模式与机制[J].情报理论与实践,2017,40(11):45-51.
	Li C Z,Zhang L L,Hou Y F,et al.Scientific Big Data Opening and Sharing:Models and Mechanisms[J].Information Studies:Theory & Application,2017,40(11):45-51.
9	张丽丽,黎建辉.科研数据的开放:进展、模式与新探索[J].大数据,2016(6):25-33.
	Zhang LL,Li J H.Research Data Openness:Development, Models and New Exploration[J].Big Data Research,2016(6):25-33.
10	Features Figshare [EB/OL]. [2020-02-10]..
11	Hahnel M.Exclusive：figshare a new open data project that wants to change the future of scholarly publishing[EB/OL]. [2020-02-10]..
12	Dryad[EB/OL]. [2020-02-10]..
13	Vision T.The Dryad Digital Repository: Published evolutionary data as part of the greater data ecosystem[J].Nature Precedings,2010:1-1.
14	ScienceDB[EB/OL]. [2020-02-10]..
15	DataONE[EB/OL]. [2020-02-10]..
16	Michener W,Vieglais D,Vision T,et al.DataONE: Data Observation Network for Earth—Preserving Data and Enabling Innovation in the Biological and Environmental Sciences[J].D-Lib Magazine,2011,17(1/2):12.
17	Michener W K,Allard S,Budden A,et al.Participatory design of DataONE—enabling cyber infrastructure for the biological and environmental sciences[J].Ecological Informatics,2012,11:5-15.
18	Christian E J.GEOSS Architecture Principles and the GEOSS Clearinghouse[J].IEEE Systems Journal,2008,2(3):333-337.
19	Group on Earth Observations.GEOSS 10-Year Implementation Plan[EB/OL]. [2020-02-10]..
20	Bai Y,Di L,Nebert D D,et al.GEOSS Component and Service Registry: Design, Implementation and Lessons Learned[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2012,5(6):1678-1686.
21	中国科学院数据云[EB/OL]. [2020-02-10]..
	Data Cloud of CAS[EB/OL]. [2020-02-10]..
22	黎建辉,周园春,胡良霖,等.中国科学院科学数据云建设与服务[J].大数据,2016,2(6):3-13.
	Li J H,Zhou Y C,Hu L L,et al.Construction and service of scientific data cloud of Chinese Academy of Sciences[J].Big Data Research,2016,2(6):3-13.
23	Landsat Data Access[EB/OL]. [2020-02-10]..
24	Irons J R,Dwyer J L,Barsi J A.The Next Landsat Satellite: The Landsat Data Continuity Mission[J].Remote Sensing of Environment,2012,122:11-21.
25	ESA.Sentinel Online Data Access[EB/OL]. [2020-02-10]..
26	Torres R,Snoeij P,Geudtner D,et al.GMES Sentinel-1 Mission[J].Remote Sensing of Environment,2012,120:9-24.
27	LAMOST Data Access[EB/OL]. [2020-02-10]..
28	Zhao G,Zhao Y H,Chu Y Q,et al.LAMOST Spectral Survey—An Overview[J].Research in Astronomy and Astrophysics,2012,12(7):723.
29	Luo A L,Zhang H T,Zhao Y H,et al.Data release of the LAMOST pilot survey[J].Research in Astronomy and Astrophysics,2012,12(9):1243.
30	SDSS Data Access[EB/OL]. [2020-02-10]..
31	Eisenstein D J,Weinberg D H,Agol E,et al.SDSS-III: Massive Spectroscopic Surveys of the Distant Universe, the Milky Way, and Extra-solar Planetary Systems[J].The Astronomical Journal,2011,142(3):72.
32	地理空间数据云[EB/OL]. [2020-02-10]..
	Geospatial Data Cloud[EB/OL]. [2020-02-10]..
33	国家生态系统观测研究网络[EB/OL]. [2020-02-10]..
	National Ecosystem Research Network of China[EB/OL]. [2020-02-10]..
34	中国生态系统网络[EB/OL]. [2020-02-10]..
	Chinese Ecosystem Research Network[EB/OL]. [2020-02-10]..
35	郭学兵,苏文,唐新斋,等.云计算环境下CNERN资源管理与服务平台的构建[J].中国科技资源导刊,2017,49(1):30-37.
	Guo X B,Su W,Tang X Z,et al.Construction of CNERN Resource Management and Service Platform in Cloud Computing Environment[J].China Science & Technology Resources Review,2017,49(1):30-37.
36	Lewis A,Oliver S,Lymburner L,et al.The Australian geoscience data cube—foundations and lessons learned[J].Remote Sensing of Environment,2017,202:276-292.
37	Kopp S,Becker P,Doshi A,et al.Achieving the Full Vision of Earth Observation Data Cubes[C].International Conference on Data Technologies and Applications,2019,4(3):94.
38	柏永青,杨雅萍,孙九林.国内外科学数据管理办法研究进展[J].农业大数据学报,2019,1(3):5-20.
	Bai Y Q,Yang Y P,Sun J L.Advance in the Study of Domestic and Foreign Data Management Methods[J].Journal of Agricultural Big Data,2019,1(3):5-20.
39	温亮明,张丽丽,黎建辉.大数据时代科学数据共享伦理问题研究[J].情报资料工作,2019,40(2):38-44.
	Wen L M,Zhang L L,Li J H.Research on Ethical Issues of Scientific Data Sharing in the Big Data Era[J].Information and Documentation Services,2019,40(2):38-44.
40	Bica M,Bacu V,Mihon D,et al.Architectural Solution for Virtualized Processing of Big Earth Data[C].IEEE International Conference on Intelligent Computer Communication & Processing,2014.DOI：10.1109/ICCP.2014.6937027. doi: 10.1109/ICCP.2014.6937027
41	程学旗,靳小龙,王元卓,等.大数据系统和分析技术综述[J].软件学报,2014,25(9):1889-1908.
	Cheng X Q,Jin X L,Wang Y Z,et al.Survey on Big Data System and Analytic Technology[J].Journal of Software,2014,25(9):1889-1908.
42	Killough B.Overview of the Open Data Cube Initiative[C].2018 IEEE International Geoscience and Remote Sensing Symposium(IGARSS2018,2018:8629-8632.
43	Karpatne A,Liess S.A Guide to Earth Science Data:Summary and Research Challenges[J].Computing in Science and Engineering,2015,17(6):14-18.
44	Camara G,De Assis L F,Ribeiro G,et al.Big Earth Observation Data Analytics:Matching Requirements to System Architectures[C].International Workshop on Analytics for Big Geospatial Data,2016:1-6.
45	Guo H D,Liu Z,Jiang H,et al.Big Earth Data:A New Challenge and Opportunity for Digital Earth's Development[J].International Journal of Digital Earth,2017,10(1):1-12.
46	Fouad K,Bixby J L,Callahan A,et al.FAIR SCI Ahead:The Evolution of the Open Data Commons for Preclinical Spinal Cord Injury Research[J].Journal of Neurotrauma,2019,.
47	张丽丽,温亮明,石蕾,等.国内外科学数据管理与开放共享的最新进展[J].中国科学院院刊,2018,33(8):774-782.
	Zhang L L,Wen L M,Shi L,et al.Progress in Scientific Data Management and sharing[J].Bulletin of Chinese Academy of Sciences,2018,33(8):774-782.
48	Wilkinson M D,Dumontier M,Aalbersberg I J,et al.The FAIR Guiding Principles for Scientific Data Management and Stewardship[J].Scientific Data,2016,3(1):160018.
49	Wilkinson M D,Sansone S,Schultes E,et al.A Design Framework and Exemplar Metrics for FAIRness[J].Scientific Data,2018,5(1):180118.
50	Mons B,Schultes E,Liu F H,et al.The FAIR Principles:First Generation Implementation Choices and Challenges[J].Data Intelligence,2019,2(1/2):1-9.
51	Aiello G,Giovino I,Vallone M,et al.A Decision Support System Based on Multisensor Data Fusion for Sustainable Greenhouse Management[J].Journal of Cleaner Production,2018:4057-4065.
52	Stadnikia K,Henderson K,Koppal S J,et al.Data Fusion for a Vision-aided Radiological Detection System:Correlation Methods for Single Source Tracking[J].Nuclear Instruments & Methods in Physics Research Section A-accelerators Spectrometers Detectors and Associated Equipment,2019.DOI:10.1016/j.nima.2019.02.040. doi: 10.1016/j.nima.2019.02.040
53	Khankalantary S,Rafatnia S,Mohammadkhani H,et al.An Adaptive Constrained Type-2 Fuzzy Hammerstein Neural Network Data Fusion Scheme for Low-cost SINS/GNSS Navigation System[J].Applied Soft Computing,2020.DOI:10.1016/j.asoc.2019.105917. doi: 10.1016/j.asoc.2019.105917
54	Lnenicka M,Komarkova J.Big and Open Linked Data Analytics Ecosystem:Theoretical Background and Essential Elements[J].Government Information Quarterly,2019,36(1):129-144.
55	Weigel T,Schwardmann U,Klump J,et al.Making Data and Workflows Findable for Machines[J].Data Intelligence,2019,2(1/2):30-39.
56	Wen L M,Zhang L L,Li J H.Application of Blockchain Technology in Dada Management:Advantages and Solutions[J].Lecture Notes in Computer Science,2019(11473):239-254.
57	Martin S,Turki S,Renault S.Open Data Ecosystems[C].Electronic Government and the Information Systems Perspective:6th International Conference,2017:49-63.DOI:10.1007/978-3-319-64248-2_5. doi: 10.1007/978-3-319-64248-2_5
58	Zuiderwijk A,Janssen M,Davis C,et al.Innovation with Open Data:Essential Elements of Open Data Ecosystems[J].Information polity,2014:17-33.
59	Iury S Oliveira Marcelo,Bernadette Farias Lóscio.What is a Data Ecosystem[C].Proceedings of the 19th Annual International Conference on Digital Government Research,2018:1-9..
60	郭华东.地球大数据科学工程[J].中国科学院院刊,2018,33(8):818-824.
	Guo H D.A project on Big Earth Data Science Engineering[J].Bulletin of Chinese Academy of Sciences,2018,33(8):818-824.
61	Yang C W,Yu M Z,Li Y,et al.Big Earth Data Analytics: A Survey[J].Big Earth Data,2019,3, (2):83-107.

系统	所属机构	服务对象	提交方式	标识符	许可协议	质量控制
Figshare	英国Digital Science公司	科研人员、机构、出版商	门户网站上传、桌面工具、API	DOI	CC0	不提供标准的数据审核机制，由提交者自行核查
Dryad	美国国家科学基金会	科研人员、机构、出版商	门户网站上传、API	DOI	CC0	系统管理员进行数据质量审核
ScienceDB	中国科学院	科研人员、出版商、科研项目、机构高校	门户网站上传、FTP工具、线下拷贝	DOI、CSTR	CC0、CC BY 4.0	提供数据质量审核

系统类型	数据来源	主要目标	核心服务	适用场景	缺陷不足
仓储型系统	来源于用户提交	收集科研人员的最新数据文件进行共享发布，形成科学数据新型出版模式	数据共享、数据出版	使用者希望发布数据资源或搜索最新科研数据	数据来源无法保证，数据质量参差不齐
联邦服务型系统	分散在各地的数据节点	逻辑上汇聚多源数据，形成统一资源目录，方便对外共享	数据汇聚、数据共享	使用者对数据类型、范围、体量有较大需求	数据资源散落分布，易受数据所有者影响
数据分发型系统	大型科学装置、科学实验、监测站点	提供领域内专业规范的数据资源，促进数据共享和重用	数据检索、数据获取	使用者对数据质量有多维度、细粒度需求	数据来源和类型较单一，受到行业领域性质局限
按需计算与分析的云服务型系统	用户个人提交或公开数据集获取	提供稳定的云端计算环境，通过多种交互分析模式快速进行数据计算	数据分析	使用者本身缺乏计算工具和环境，但关注分析结果而对具体处理流程要求较低	对数据质量和用户专业性要求较高，对多源异构数据分析处理较困难

[1]	段博文, 王卷乐, 石蕾, 高孟绪. 前沿领域国内外典型数据库调研与启示[J]. 农业大数据学报, 2023, 5(1): 46-54.
[2]	王鑫, 郭雷风. 防返贫监测大数据融合框架的构建与应用[J]. 农业大数据学报, 2022, 4(2): 108-118.
[3]	孙雨潇, 李艳丽, 李峰, 陈谦. 国内外科学数据共享现状研究与发展建议[J]. 农业大数据学报, 2022, 4(2): 88-98.
[4]	薛沐涵, 徐硕, 刘慧媛, 鲁峰, 王宇, 李奥. 远洋渔业科学数据治理与应用服务研究[J]. 农业大数据学报, 2022, 4(2): 99-107.
[5]	王亚鹏, 张文革, 胡林, 刘婷婷, 曹姗姗, 王蕾, 孙伟. 2019年天山云杉背包式激光雷达三维参数测量数据集[J]. 农业大数据学报, 2022, 4(1): 119-124.
[6]	吴迪, 汪勇, 孙地冰, 李杰, 陈昆松. 新冠疫情下港区农产品冷链物流监管平台构建与应用[J]. 农业大数据学报, 2022, 4(1): 62-68.
[7]	范亚慧, 朱亮, 赵华, 郑建华. 科学数据共享的知识产权保护研究[J]. 农业大数据学报, 2021, 3(4): 3-9.
[8]	薛沐涵, 徐硕, 鲁峰, 朱勇, 吴建光, 王义刚. 渔船渔港综合管理服务平台构建与应用[J]. 农业大数据学报, 2021, 3(3): 45-54.
[9]	冯小鼎, 王晓冬, 罗斌, 王成. 基于LabVIEW的植物离子吸收多参数检测系统软件研发[J]. 农业大数据学报, 2021, 3(2): 16-23.
[10]	李强, 高懋芳, 方莹. 农业大数据信息平台构建方法初探[J]. 农业大数据学报, 2021, 3(2): 24-30.
[11]	颜瑞, 王震, 李言浩, 李哲敏, 李娴. 中国农业智能传感器的应用、问题与发展[J]. 农业大数据学报, 2021, 3(2): 3-15.
[12]	王慧, 王海江, 高攀, 张泽, 侯彤瑜, 吕新. 新疆生产建设兵团农业资源数据采集与整合方法研究[J]. 农业大数据学报, 2021, 3(2): 31-41.
[13]	王晓丽, 胡乾浩, 满芮, 刘婷婷. 中国果树花期2016年近红外光谱和图像数据集[J]. 农业大数据学报, 2021, 3(1): 88-93.
[14]	庄严, 杨帅, 刘照坤, 樊景超, 周舒雅. 农业科学观测数据权属与保护路径研究[J]. 农业大数据学报, 2020, 2(4): 107-112.
[15]	王晓丽, 胡乾浩, 樊景超, 李壮. 辽北苹果叶片氮含量、近红外光谱与图像数据集[J]. 农业大数据学报, 2020, 2(4): 113-119.