Software Developed in Brazil Prepares to Orchestrate Data from Telescope with World’s Largest Digital Camera

Kytos, a Software Defined Networking (SDN) platform developed by SPRACE, is being used in AmLight’s production network. The AmLight consortium will be responsible for the international data transmission of the LSST telescope.

Since December 2018, the AmLight consortium, which will be responsible for the Large Synoptic Survey Telescope (LSST) international data transmission, is using Kytos in its production network. The AmLight consortium members include Florida International University (FIU), the Academic Network of Sao Paulo (ANSP), the National Research and Education Network (NREN) of Brazil (RNP), the regional network of Latin America (RedCLARA), the Association of Universities for Research in Astronomy (AURA), Florida LambdaRail (FLR), and Internet2. Kytos is an open source platform for orchestration of Software Defined Networking (SDN), developed by the São Paulo Research and Analysis Center (SPRACE), with financial support from Brazil’s Lei da Informática. The project led to a close collaboration with engineers and developers from the Academic Network of São Paulo (ANSP), Rede Nacional de Ensino e Pesquisa (RNP), and Florida International University.

SDN technology allows network devices to be controlled by software, enabling greater flexibility and better network management, avoiding vendor lock-in. The Kytos platform was created to provide a fast way to deploy an SDN network. This can be accomplished through a plug-and-play solution or the development of a customized controller. The platform was designed to be easy to install and use, and it allows the development and sharing of network applications by the user community.

The Kytos SDN controller was evaluated on several national and international testbeds. During demonstrations at SuperComputing 2017, the most important conference in the area of high-performance computing and networking, Kytos was responsible for orchestrating an intercontinental network ring using AmLight’s Atlantic and Pacific 100G links. Total bandwidth utilization of 350 Gbps in data transfers was observed during the demonstration. During this demonstration, the Kytos team was invited to deploy and test the Kytos software on the Energy Sciences Network (ESNet) testbed, a high-speed network that connects more than 40 institutions linked to the Department of Energy (DoE). The current objective is to prepare Kytos to meet the demands of the Large Synoptic Survey Telescope project, which will be the telescope with the world’s largest digital camera.

The LSST telescope is an international project proposed in 2001 that began to be officially built in 2014. The full LSST operation is scheduled for 2022. The chosen site for construction was Cerro Pachón, in Chile, at the peak of a 2600 meters mountain. The LSST will photograph all visible southern sky, which entails significant challenges in the areas of transmission, storage, and data analysis. Each night about 20 TB of data will be generated and transmitted to different analysis centers around the world, including the National Center for Supercomputing Applications (NCSA), in Illinois, to where all data will be first sent in real time. After ten years of operations, the project is expected to process hundreds of Petabytes (PB), from which a database of more than 15 PB of information about the universe will be generated.

The AmLight consortium will be responsible for international data transmission from Chile to the USA. To transfer large amounts of information quickly between countries, AmLight will use a complex network with multiple 100 Gbps links. The consortium will also have in its infrastructure Software-Defined Networking to lower operating costs and support complex and dynamic services.

Kytos’ development team emphasizes that the platform has been developed focusing on large-scale scientific experiments, such as the Compact Muon Solenoid (CMS), one of the particle detectors of CERN’s Large Hadron Collider (LHC), and the Large Synoptic Survey Telescope. “It is a great pleasure to see a SPRACE initiative, that was initially focused on meeting demands of High Energy Physics, surpassing frontiers and reaching other areas of e-Science,” says Sérgio Novaes, principal investigator at SPRACE.

The fact that Kytos is being incorporated into AmLight’s production network demonstrates the potential of the platform and represents an important step towards its incorporation into the LSST project. “Kytos is not being developed only by the SPRACE research group anymore, as it took international proportions. As an open platform, collaborators from various groups are concentrating efforts to deliver a robust and reliable solution. The engagement of the AmLight engineers and Kytos developers is fundamental for the project to be used in a production environment, in the Brazil-USA links. This partnership shows that we are on the right track to achieve the maturity required for a project of the size of LSST”, explains Beraldo Leal, Kytos development team coordinator.

Acknowledgments
AmLight Express & Protected is funded by the International Research Network Connections (IRNC) through Award # ACI-1451018.
The LSST is funded by the National Science Foundation (NSF), the US Department of Energy (DOE) and privately funded by the LSST Corporation.

More information

About SPRACE
The São Paulo Research and Analysis Center (SPRACE) develops research in the areas of basic science, high-performance computing, and digital innovation. Created in 2003 with the support of the São Paulo Research Foundation (FAPESP), SPRACE enables the participation of High Energy Physics researchers from the State of São Paulo in the Compact Muon Solenoid (CMS) collaboration of the Large Hadron Collider  (LHC), the particle accelerator of the European Organization for Nuclear Research (CERN). As part of its role in the LHC, SPRACE operates a Tier 2 computational structure (BR-SP-SPRACE) of the LHC Computing Grid (WLCG), which processes, analyzes and stores part of the data generated by CMS. SPRACE engineers work on the field of scientific instrumentation by developing a Level 1 trigger that will be implemented in the second phase of the CMS tracking detector upgrade. SPRACE has partnerships with the private sector to develop digital innovation projects, with an emphasis on High-Performance Computing (HPC), Machine Learning and Software Defined Networks (SDN). Its team developed Kytos, an open-source SDN platform for network control that is being used in AmLight’s international link. For more information about SPRACE, visit our website at https://www.sprace.org.br/.

For a copy of the article on Portuguese click here.