Research

Targeting new use cases and applications (O1), one of the main focuses was on a special type of eXtended Reality (XR), namely, Mixed Reality (MR) applications, where the interaction between real and virtual objects is allowed. One such concrete application that we have implemented for demonstration purposes is a multi-user game where a physical toy car can be controlled arbitrarily with a physical remote controller, which can carry a virtual box on top and various virtual objects can be thrown into it while moving. In this case, virtual objects connected to dynamically changing physical elements and environments must be displayed to users while the virtual and physical components are constantly synchronized and different users can see their corresponding own views on their own devices. These services, implemented as distributed software, with some components running on the MR device and other modules running in the cloud (or in the edge), impose very special demands on the underlying network and cloud infrastructure if we want to provide an acceptable user experience. Using the specific application (and its different variants), we analyzed in detail the effects of latency, jitter, uplink / downlink bandwidth, packet loss, and available computing capacity on the user experience and we identified minimum requirements. The prototype was demonstrated at internal events of the industry partner.

In addition, we continued our work with our multi-player Augmented Reality (AR) application, where the game engine and the SLAM (Simultaneous Localization And Mapping) module, which is responsible for the continuous positioning of the AR devices, can be deployed to cloud or edge cloud environment and the synchronization and coordination among the players also take place from there. The results are summarized in a submitted conference paper. Besides, a number of BSc theses have been prepared studying different aspects of human-robot collaboration and AR/VR applications.

Another important application type addressed during the reporting period is “telco,” the realm of classic telecommunications services. These traditional telco services (e.g. voice) provided in the next generation network environment (e.g. 5G and beyond systems) are still of great importance today and their quality requirements are very specific. During the pandemic, we have seen how essential video conferencing has become, and its quality fundamentally affects their usability and the workflows based on them. In our work, we examined in detail the constraints and requirements on cloud and network platforms and on the applied computing paradigms which stem from the special features and quality requirements of these applications.

During the first three years, multiple use cases were investigated from the fields of cloud robotics (controlling e.g. robots, drones or other unmanned vehicles from the cloud), Virtual Reality (VR), Augmented Reality (AR). All these applications pose serious challenges to the underlying network and cloud infrastructures including strict latency requirements, ultra high reliability and availability.

First, we addressed the distributed control of indoor and outdoor drones from an integrated system of networks and clouds and we investigated how to run applications with strict delay requirements in a distributed cloud environment. We have developed several solutions, where the control software is divided into several software components with different resource requirements and QoS (Quality of Service) constraints, which can be run in different environments (cloud, edge) if we can ensure the proper communication between them. On the one hand, we have implemented an SFC-based (Service Function Chain) solution where the application is constructed form connected Virtual Network Functions (VNF). This solution builds on ETSI’s NFV architecture and follows a compatible approach. The application has been integrated with our own orchestration framework and the results were demonstrated at the most important forum on the field, at ACM SIGCOMM 2018 [2018-1]. On the other hand, we have examined how similar applications, such as the controller of an unmanned rover [2020-1], can be implemented based on the relatively new Function as a Service (FaaS) concept, and whether the platform is capable of provisioning services with strict delay requirements. Furthermore, we elaborated a microservice-based object detection software component suitable for outdoor drones (or other unmanned vehicles) and we explored different deployment options on public and private cloud platforms [2019-2].

Second, we designed and implemented the components needed to support human-robot collaboration (HRC) in future production environments, and explored how these software could be adapted to the latest cloud platforms and cloud services (e.g., serverless computing). The main goals were to investigate the impact of the extra latency and jitter introduced by the cloud platforms on the performance of the applications, and to identify the benefits provided by the clouds in software development and operation.

Third, we focused on the different types of extended reality (AR / VR / MR, Augmented / Virtual / Mixed Reality) applications. On the one hand, we analyzed the effects of increased latency and the benefits of automatic resource scaling (vertical and horizontal ones). We established a dedicated virtual reality environment for future production systems and the previous HRC application was implemented and evaluated in this space. An integrated demo including the HRC and VR components (human-robot collaboration in virtual space) were presented at IEEE ICNP 2019 conference [2019-1]. On the other hand, we elaborated a multi-player AR gaming use case where the distributed software components, including the game server and the SLAM (Simultaneous Localization And Mapping) module, can be deployed to different platforms or runtime environments (e.g., edge cloud, central cloud, AR device). The main features and benefits of our approach were demonstrated at several internal events of our industry partner and a conference paper is planned for the next year.

We also addressed the business aspects of the whole ecosystem. In this regard, we modeled and analyzed a federation of 5G infrastructure providers who aims at selecting the compute and network resource set that fulfills the technical requirements of the service deployment and that is preferable also from an economic point of view. We modeled this resource market with the tool set of graph and game theory and studied its characteristics. We also derived the best pricing strategies that the providers should follow given their expectation about customers’ demand. Our main results are summarized in a journal paper [2020-2].

In another work [2020-3], we analyzed federations of telcos, cloud administrators, and online application suppliers who unite for conveying future services to clients around the world. In order to help their portability, or the simple geographic range of the offered application, the business arrangements among the actors must scale over numerous domains and a guaranteed nature of joint effort among different stakeholders is important. In this environment, business perspectives will significantly impact the technical capacity of the system: the business arrangements of the providers will innately decide the accessibility and the end-client costs of certain services. In this work, we modeled the business relations of providers as a variation of network formation games. We inferred conditions under which the current transit-peering structure of network providers stays unblemished, and we also drew the specifics of an envisioned setup in which providers create business links among each other starting with a clean slate.

• [2018-1] J. Czentye, J. Dóka, Á. Nagy, L. Toka, B. Sonkoly, R. Szabó, “Controlling Drones from 5G Networks,” In Proc. of ACM SIGCOMM 2018 (Demo)

• [2019-1] B. Gy. Nagy, J. Dóka, S. Rácz, G. Szabó, I. Pelle, J. Czentye, L. Toka, B. Sonkoly, “Towards Human-Robot Collaboration: An Industry 4.0 VR Platform with Clouds Under the Hood,” IEEE ICNP 2019 (Demo)

• [2019-2] J. Dóka, “Elaboration of a latency critical future 5G application and adaptation to a selected FaaS platform”, MSc Thesis, BME VIK 2019 (in Hungarian)

• [2020-1] I. Pelle, F. Paolucci, B. Sonkoly, F. Cugini, “Telemetry-Driven Optical 5G Serverless Architecture for Latency-Sensitive Edge Computing”, Optical Fiber Communication Conference (OFC) 2020

• [2020-2] L. Toka, M. Zubor, A. Kőrösi, G. Darzanos, O. Rottenstreich, B. Sonkoly, “Pricing games of NFV infrastructure providers”, Telecommunication Systems, doi: 10.1007/s11235-020-00706-5, 2020 (IF: 1.73)

• [2020-3] L. Toka, A. Recse, M. Cserép, R. Szabó, “On the Mediation Price War of 5G Providers”, MDPI Electronics, 9:11, 1901 pp. 1-19, 2020 (IF: 2.41)

As novel use-cases, two new areas were investigated. On the one hand, we designed and implemented the components needed to support human-robot collaboration in future production environments, and explored how these software could be adapted to the latest cloud platforms and cloud services (e.g., serverless computing). The main goals were to investigate the impact of the extra latency and jitter introduced by the cloud platforms on the performance of the applications, and to identify the benefits provided by the clouds in software development and operation. On the other hand, we focused on the different types of augmented reality (AR / VR / MR, Augmented / Virtual / Mixed Reality) applications, and more specifically, we analyzed the effects of increased latency and the benefits of automatic resource scaling (vertical and horizontal ones). The two application examples were integrated in a common demo (human-robot collaboration in virtual space) and presented at IEEE ICNP 2019 conference [2019-1]. In addition, we elaborated a microservice-based object detection software component suitable for outdoor drones (or other unmanned vehicles) and we explored different deployment options on public and private cloud platforms [2019-2]. We also addressed the business aspects of the whole ecosystem. In this regard, we modeled and analyzed a federation of 5G infrastructure providers who aims at selecting the compute and network resource set that fulfills the technical requirements of the service deployment and that is preferable also from an economic point of view. We modeled this resource market with the tool set of graph and game theory and studied its characteristics. We also derived the best pricing strategies that the providers should follow given their expectation about customers' demand.

• [2019-1] B. Gy. Nagy, J. Dóka, S. Rácz, G. Szabó, I. Pelle, J. Czentye, L. Toka, B. Sonkoly, “Towards Human-Robot Collaboration: An Industry 4.0 VR Platform with Clouds Under the Hood,” IEEE ICNP 2019 (Demo)

• [2019-2] J. Dóka, “Elaboration of a latency critical future 5G application and adaptation to a selected FaaS platform”, MSc Thesis, BME VIK 2019 (in Hungarian)

As a new use-case (O1), we addressed the distributed control of indoor and outdoor drones from an integrated system of networks and clouds and we investigated how to run applications with strict delay requirements in a distributed cloud environment. We have developed several solutions, where the control software is divided into several software components with different resource requirements and QoS (Quality of Service) constraints, which can be run in different environments (cloud, edge) if we can ensure the proper communication between them. On the one hand, we have implemented an SFC-based (Service Function Chain) solution where the application is constructed form connected Virtual Network Functions (VNF). This solution builds on ETSI's NFV architecture and follows a compatible approach. The application has been integrated with our own orchestration framework and the results were demonstrated at the most important forum on the field, at ACM SIGCOMM 2018 [2018-1]. On the other hand, we have examined how similar applications can be implemented based on the relatively new Function as a Service (FaaS) concept, and whether services with strict delay requirements in this environment can be implemented.

• [2018-1] J. Czentye, J. Dóka, Á. Nagy, L. Toka, B. Sonkoly, R. Szabó, “Controlling Drones from 5G Networks,” In Proc. of ACM SIGCOMM 2018 (Demo)

During the reporting period, several work started earlier was completed and published in the relevant journals after the required revision cycles. One of our most important results is our survey article published in IEEE Communications Surveys & Tutorials (COMST) [2021-3], in which we reviewed and analyzed the literature on placement methods proposed for the edge cloud environment based on a newly developed taxonomy. In addition, we have revised and successfully published an article summarizing our proposed and implemented VNF (Virtual Network Function) deployment algorithms for mobile and volatile 5G infrastructures [2021-4].

In the current period, our main goal was to develop new orchestration and control methods to support latency-sensitive applications in a cloud native environment where software can take advantage of modern cloud technologies such as the serverless paradigm and the Function as a Service (FaaS) computing model. We proposed several theoretical solutions and working prototypes [2021-5], [2021-6], [2021-7]. Our long-term goal is to develop a platform that also supports real-time applications with strict operational guarantees. The first important result of this work is our paper published at the IEEE CLOUD conference (honored with Best Paper Award), which provides real-time task scheduling in a FaaS environment [2021-8].

In addition, we designed and implemented new automatic scaling solutions for telco applications that can be used in a cloud native environment. On the one hand, we proposed a horizontal scaling mechanism combining several machine learning algorithms for a Kubernetes-managed edge cloud environment and evaluated its performance in different operating ranges. The results were summarized in an IEEE Transactions on Network and Service Management (TNSM) journal article [2021-9]. Furthermore, we proposed a method for optimizing cloud resources that is able to operate application components (typically telco) cost-effectively based on a new, proactive horizontal scaling approach based on learning user behavior. The proposed method was described in an article submitted to the IEEE ICC Conference. We have also designed a “data lake”-based platform for reliable operation of cloud native telco applications, which can monitor all relevant parameters of the cloud infrastructure and extract and derive various types of information from them, as well as predict failures, which is essential in production environment. The main results were presented in an article published at the IFIP / IEEE International Symposium on Integrated Network Management (IM) [2021-10].

• [2021-3] B. Sonkoly, J. Czentye, M. Szalay, B. Németh, L. Toka, “Survey on Placement Methods in the Edge and Beyond”, IEEE Communications Surveys and Tutorials (COMST), 23:(4), pp. 2590-2629., 2021 (IF: 25.249)

• [2021-4] B. Németh, N. Molner, J. Martin-Perez, C. J. Bernardos, A. de la Oliva, B. Sonkoly, “Delay and reliability-constrained VNF placement on mobile and volatile 5G infrastructure”, IEEE Transactions on Mobile Computing (TMC), 2021, DOI: 10.1109/TMC.2021.3055426 (IF: 5.577)

• [2021-5] L. Toka, “Ultra-Reliable and Low-Latency Computing in the Edge with Kubernetes”, Springer Journal of Grid Computing, 19:(3), 2021 (IF: 3.986)

• [2021-6] M. Szalay, P. Mátray, L. Toka, “State Management for Cloud-Native Applications”, MDPI Electronics, 10:(4), 2021 (IF: 2.397)

• [2021-7] D. Haja, Z.R. Turanyi, L. Toka, “Location, Proximity, Affinity – The key factors in FaaS”, Infocommunications Journal, 12:(4), pp. 14-21, 2021

• [2021-8] M. Szalay, P. Mátray, L. Toka, “Real-time task scheduling in a FaaS cloud”, in Proc. of IEEE International Conference on Cloud Computing (CLOUD), 2021

• [2021-9] L. Toka, G. Dobreff, B. Fodor, B. Sonkoly, “Machine Learning-based Scaling Management for Kubernetes Edge Clusters”, IEEE Transactions on Network and Service Management (TNSM), 18:(1), pp. 958-972., 2021 (IF: 4.195)

• [2021-10] L. Toka, G. Dobreff, D. Haja, M. Szalay, “Predicting cloud-native application failures based on monitoring data of cloud infrastructure”, in Proc. of IFIP/IEEE International Symposium on Integrated Network Management (IM), Piscataway (NJ), USA, 2021

One of the most significant part of the work carried out during the first three years was the elaboration and implementation of an orchestration system responsible for the automatic deployment of new types of network applications/services and its major algorithms. The central component of this system is the engine which maps (embeds) the applications (service chains) defined as inter-connected software components to available resources while fulfilling the specified (e.g., delay, bandwidth) requirements. This problem is a generalized, more complex version of VNE (Virtual Network Embedding) which is an NP-hard problem. In order to have practical solutions, we have developed various heuristic algorithms and evaluated them from different aspects, and compared them with MILP-based solutions. A special version of the algorithm was adjusted to softwarized data plane where the low-level data plane resources are orchestrated efficiently according to given performance requirements. The results were published at the IEEE INFOCOM 2018 conference with a detailed complexity analysis [2018-2]. That year at the INFOCOM conference, which is a top tier forum in our field, we appeared with an accepted demonstration [2018-6] where we showcased the self-healing feature of our orchestration system, and we also published three workshop papers. On the one hand, we have provided a hybrid algorithm that combines an online heuristic algorithm with an offline ILP-based solution [2018-3]. Basically, the system works on the basis of the online algorithm, but it runs the offline optimization periodically in the background, and then reconfigures the system at certain intervals, moving the operation state toward the global optimum. In the next paper, we have examined the business aspects of cooperating 5G operators [2018-5]. In the third paper [2018-4], we have developed an extension to OpenStack, which is one of the most common VIM (Virtual Infrastructure Manager) platforms, in order to enable the consideration of network properties and capabilities during the orchestration. As a result, OpenStack can be used for Cloud Edge resource management, where not only the central data center has computing resources but also the edge of the networks (e.g., in base stations). The proof-of-concept prototype of our OpenStack orchestrator was presented at IEEE INFOCOM 2019 conference [2019-6]. Our complete orchestration framework, which supports multi-operator operation and automatic consideration of several constraints and requirements, such as end-to-end delay, bandwidth requirements, affinity, anti-affinity, and link anti-affinity, is summarized in a journal paper published in a JSAC special issue [2020-6]. The developed solutions and algorithms are expected to be incorporated into the products of our industry partner.

In addition, in a dedicated journal paper [2020-7], we compared two architecture options together with proof-of-concept prototypes and corresponding embedding algorithms, which enable the provisioning of delay-sensitive IoT applications. On the one hand, we extended the VIM itself with network-awareness and on the other hand, we proposed a multi-layer orchestration system where an orchestrator is added on top of VIMs and network controllers to integrate different resource domains. We implemented fully-fledged solutions and conducted large-scale experiments to reveal the scalability characteristics of both approaches. We found that our VIM extension can be a valid option for single-provider setups encompassing even 100 edge domains and serving a few hundreds of customers. Whereas, our multi-layer orchestration system showed better scaling characteristics in a wider range of scenarios at the cost of a more complex control plane including additional entities and novel APIs.

As a specific complement to our orchestration systems, we defined novel substrate models capturing the characteristics and dynamicity of the rapidly evolving communication infrastructure. These models were incorporated into our placement algorithms, by these means providing novel capabilities and management options to operators. The main achievements are summarized in an MDPI Sensors journal paper [2020-8].

Besides OpenStack, which operates virtual machines, we also considered container management systems. We extended today’s most widely used container orchestration system, named Kubernetes, with the capability of taking the network characteristics into consideration in its scheduler, which makes it suitable for Edge resource management, as well. The results were demonstrated at ACM SIGCOMM 2019 [2019-7] which is a flagship conference in our research field, and an extended version of the system was described in [2019-8]. We proposed another important extension to Kubernetes in order to enhance its scaling engine. More specifically, we designed and implemented a novel, proactive horizontal auto-scaling engine which is capable of handling the actual variability of incoming requests making use of various AI-based forecast methods. In our implementation, these methods compete with each other via a short-term evaluation loop in order to always give the lead to the method that suits best the actual request dynamics. We found that our auto-scaling engine results in significantly less lost requests with slightly more provisioned resources compared to the default baseline. The first results were published at IEEE/ACM CCGRID 2020 [2020-9] which is a top-tier conference, while an extended version of the work was submitted to IEEE Transactions on Network and Service Management (TNSM).

A number of theoretical publications have also been published or submitted focusing on different aspects of the orchestration system. First, we investigated how the new orchestration methods can be applied to Big Data systems operating in geographically distributed environments [2019-9], [2019-10]. Second, we have been working in a close cooperation with our industry partner on how to efficiently manage application-related data in a serverless environment and dynamically optimize their placement based on current needs and traffic situations. The results were published at IEEE CLOUD 2019 [2019-11] and IEEE CloudNet 2019 [2019-12] conferences. Moreover, a working prototype was also implemented [2020-10] and demonstrated at ACM SIGCOMM 2020 [2020-11]. Third, we addressed a special type of VNF (Virtual Network Function) placement. More exactly, we focused on the temporal aspects and conflicting traits of reliable, low latency service deployment over a volatile network, where mobile compute nodes act as an extension of the cloud and edge computing infrastructure. This means that we can run VNFs on continuously moving mobile devices, such as robots or drones. The problem was formulated as a cost-minimizing VNF placement optimization and an efficient heuristic was also proposed. The algorithms were extensively evaluated from various aspects by simulation on detailed real-world scenarios. The achievements were summarized in a journal paper submitted to IEEE Transactions on Mobile Computing (TMC). And finally, instead of central control, we proposed a wireless access sharing framework [2020-12], in which users have a say in optimizing their quality of service on the long term, and we tackled its analysis with the tool set of stochastic game theory. Our findings showed that greedy users become polite against their counterparts when the load is relatively low with the goal of preparing for situations with high load. The results were presented at IEEE ICC 2020 conference.

Last but not least, as a comprehensive summary, we compiled and submitted a survey paper to IEEE Communications Surveys & Tutorials (COMST) providing a structured taxonomy of the vast research on placement of tasks, containers, virtual machines or any other unit of computational entity that can be moved across different types of computing infrastructure. Following the given taxonomy, the research papers were analyzed and categorized according to several dimensions, such as the structure of the supported services, the problem formulation, the applied mathematical methods, and the objectives and constraints incorporated in the optimization problems. We also revealed some important research gaps in the current literature. (The paper has gone through the first round of reviews and it is currently under revision.) Furthermore, another survey paper focusing on Industrial IoT Applications enabled by 5G was also prepared during the reporting period [2020-13].

• [2018-2] B. Sonkoly, M. Szabó, B. Németh, A. Majdán, G. Pongrácz, L. Toka, “FERO: Fast and Efficient Resource Orchestrator for a Data Plane Built on Docker and DPDK,” In Proc. of IEEE INFOCOM 2018

• [2018-3] B. Németh, M. Szalay, J. Doka, M. Rost, S. Schmid, L. Toka, B. Sonkoly, “Fast and efficient network service embedding method with adaptive offloading to the edge,” In Proc. of IEEE INFOCOM (Workshops) 2018

• [2018-4] D. Haja, M. Szabo, M. Szalay, A. Nagy, A. Kern, L. Toka, B. Sonkoly, “How to orchestrate a distributed OpenStack,” In Proc. of IEEE INFOCOM (Workshops) 2018

• [2018-5] M. Cserep, A. Recse, R. Szabo, L. Toka, “Business network formation among 5G providers,” In Proc. of IEEE INFOCOM (Workshops) 2018

• [2018-6] J. Harmatos, D. Jocha, R. Szabó, B. P. Gero, B. Németh, J. Czentye, B. Sonkoly, “Self-healing in multi-constrained multi-actor virtualization orchestration,” In Proc. of IEEE INFOCOM 2018 (Demo)

• [2019-6] M. Szalay, D. Haja, J. Dóka, B. Sonkoly, L. Toka, “Turning OpenStack into a Fog Orchestrator”, IEEE INFOCOM 2019 (Demo)

• [2019-7] D. Haja, M. Szalay, B. Sonkoly, G. Pongracz, L. Toka, “Sharpening Kubernetes for the Edge”, ACM SIGCOMM 2019 (Demo)

• [2019-8] L. Toka, D. Haja, A. Korosi, B. Sonkoly, “Resource provisioning for highly reliable and ultra-responsive edge applications”, IEEE CloudNet 2019

• [2019-9] D. Haja, B. Vass, L. Toka, “Improving Big Data Application Performance in Edge-Cloud Systems”, IEEE CLOUD 2019

• [2019-10] D. Haja, B. Vass, L. Toka, “Towards making big data applications network-aware in edge-cloud systems”, IEEE CloudNet 2019

• [2019-11] M. Szalay, M. Nagy, D. Gehberger, Z. Kiss, P. Matray, F. Nemeth, G. Pongracz, G. Retvari, L. Toka, “Industrial-Scale Stateless Network Functions”, IEEE CLOUD 2019

• [2019-12] M. Szalay, P. Matray, L. Toka, “Minimizing state access delay for cloud-native network functions”, IEEE CloudNet 2019

• [2020-6] B. Sonkoly, R. Szabó, B. Németh, J. Czentye, D. Haja, M. Szalay, J. Dóka, B. Gerő, D. Jocha, L. Toka, “5G Applications from Vision to Reality: Multi-Operator Orchestration”, IEEE Journal on Selected Areas in Communications (JSAC), 38:7 pp. 1401-1416, 2020 (IF: 11.42)

• [2020-7] B. Sonkoly, D. Haja, B. Németh, M. Szalay, J. Czentye, R. Szabó, R. Ullah, K. Byung-Seo, L. Toka, “Scalable edge cloud platforms for IoT services”, Journal of Network and Computer Applications (JNCA), 170: Paper:102785, 2020 (IF: 5.57)

• [2020-8] B. Németh, B. Sonkoly, “Advanced Computation Capacity Modeling for Delay-Constrained Placement of IoT Services”, MDPI Sensors, 20:14, 3830, 2020 (IF: 3.27)

• [2020-9] L. Toka, G. Dobreff, B. Fodor, B. Sonkoly, “Adaptive AI-based auto-scaling for Kubernetes”, IEEE/ACM CCGRID 2020

• [2020-10] M. Szalay, P. Mátray, L. Toka, “AnnaBellaDB: Key-Value Store Made Cloud Native”, 16th International Conference on Network and Service Management (CNSM), 2020

• [2020-11] M. Szalay, P. Mátray, L. Toka, “AnnaBellaDB: a key value store for stateless network functions”, ACSM SIGCOMM 2020 (Demo)

• [2020-12] L. Toka, M. Szalay, D. Haja, G. Szabó, S. Rácz, M. Telek, “To boost or not to boost: a stochastic game in wireless access networks”, IEEE International Conference on Communications (ICC) 2020

• [2020-13] P. Varga, J. Pető, A. Franko, D. Balla, D. Haja, F. Janky, G. Sóos, D. Ficzere, M. Maliosz, L. Toka, “5G support for Industrial IoT Applications – Challenges, Solutions, and Research gaps”, MDPI Sensors, 20:3, 828, 2020 (IF: 3.27)

An important part of this work phase was the development and implementation of the orchestration system responsible for the automatic deployment of the novel distributed applications. One of the central components of this system is the engine which maps the applications defined as inter-connected software components to available resources while meeting the specified (e.g., delay) requirements. To solve this problem efficiently, we developed different algorithms and evaluated their performance. The proposed solutions have been implemented as part of real platforms and two important demo papers have been published. First, we added an own orchestration component to the OpenStack system which was presented at IEEE INFOCOM 2019 conference [2019-6]. Second, we extended today’s most widely used container orchestration system, named Kubernetes, with the capability of taking the network characteristics into consideration in its scheduler. As a result, it is suitable for Edge resource management. The results were demonstrated at ACM SIGCOMM 2019 [2019-7] which is a flagship conference in our research field, and an extended version of the system was described in [2019-8]. We proposed another important extension to Kubernetes in order to enhance its scaling engine. More specifically, we designed and implemented a novel, proactive horizontal auto-scaling engine which is capable of handling the actual variability of incoming requests making use of various AI-based forecast methods. In our implementation, these methods compete with each other via a short-term evaluation loop in order to always give the lead to the method that suits best the actual request dynamics. We found that our auto-scaling engine results in significantly less lost requests with slightly more provisioned resources compared to the default baseline. A paper on the results was submitted to IEEE/ACM CCGRID 2020. A number of theoretical publications have also been published or submitted focusing on different aspects of the orchestration system. First, we investigated how the new orchestration methods can be applied to Big Data systems operating in geographically distributed environments [2019-9], [2019-10]. Second, we have been working in a close cooperation with our industry partner on how to efficiently manage application-related data in a serverless environment and dynamically optimize their placement based on current needs and traffic situations. The results were published at IEEE CLOUD 2019 [2019-11] and IEEE CloudNet 2019 [2019-12] conferences. Third, we addressed a special type of VNF (Virtual Network Function) placement. More exactly, we focused on the temporal aspects and conflicting traits of reliable, low latency service deployment over a volatile network, where mobile compute nodes act as an extension of the cloud and edge computing infrastructure. This means that we can run VNFs on continuously moving mobile devices, such as robots or drones. The problem was formulated as a cost-minimizing VNF placement optimization and an efficient heuristic was also proposed. The algorithms were extensively evaluated from various aspects by simulation on detailed real-world scenarios. A paper summarizing this work was submitted to ACM Mobihoc 2020.

• [2019-6] M. Szalay, D. Haja, J. Dóka, B. Sonkoly, L. Toka, “Turning OpenStack into a Fog Orchestrator”, IEEE INFOCOM 2019 (Demo)

• [2019-7] D. Haja, M. Szalay, B. Sonkoly, G. Pongracz, L. Toka, “Sharpening Kubernetes for the Edge”, ACM SIGCOMM 2019 (Demo)

• [2019-8] L. Toka, D. Haja, A. Korosi, B. Sonkoly, “Resource provisioning for highly reliable and ultra-responsive edge applications”, IEEE CloudNet 2019

• [2019-9] D. Haja, B. Vass, L. Toka, “Improving Big Data Application Performance in Edge-Cloud Systems”, IEEE CLOUD 2019

• [2019-10] D. Haja, B. Vass, L. Toka, “Towards making big data applications network-aware in edge-cloud systems”, IEEE CloudNet 2019

• [2019-11] M. Szalay, M. Nagy, D. Gehberger, Z. Kiss, P. Matray, F. Nemeth, G. Pongracz, G. Retvari, L. Toka, “Industrial-Scale Stateless Network Functions”, IEEE CLOUD 2019

• [2019-12] M. Szalay, P. Matray, L. Toka, “Minimizing state access delay for cloud-native network functions”, IEEE CloudNet 2019

One of the most significant part of the first year's work was the elaboration and implementation of an orchestration system responsible for the automatic deployment of new types of network applications/services and its major algorithms (O3). The central element of this system is the mapping (or embedding) of the service chains specified as inputs to the available resources while fulfilling the specified requirements. This problem is a generalized, more complex version of VNE (Virtual Network Embedding) which is an NP-hard problem. In order to have practical solutions, we have developed various heuristic algorithms and evaluated them from different aspects, and compared them with MILP-based solutions. A special version of the algorithm was adjusted to softwarized data plane where the low-level data plane resources are orchestrated efficiently according to given performance requirements. The results were published at the IEEE INFOCOM 2018 conference with a detailed complexity analysis [2018-2]. At the INFOCOM conference, which is a top tier forum in our field, we also published three workshop papers. On the one hand, we have provided a hybrid algorithm that combines an online heuristic algorithm with an offline ILP-based solution [2018-3]. Basically, the system works on the basis of the online algorithm, but it runs the offline optimization periodically in the background, and then reconfigures the system at certain intervals, moving the operation state toward the global optimum. In the next paper [2018-4], we have developed an extension for OpenStack, which is one of the most common VIM (Virtual Infrastructure Manager) platforms, in order to enable the consideration of network properties and capabilities during the orchestration. As a result, OpenStack can be used for Cloud Edge resource management, where not only the central data center has computing resources but also the edge of the networks (e.g., in base stations). In the third paper we have examined the business aspects of cooperating 5G operators [2018-5]. Moreover, we appeared at this forum with an accepted demonstration as well [2018-6], where we showcased the self-healing feature of our orchestration system. The complete orchestration framework, which supports multi-operator operation and automatic consideration of several constraints and requirements, such as end-to-end delay, bandwidth requirements, affinity, anti-affinity, and link anti-affinity, is summarized in a submitted journal paper. The developed solutions and algorithms are expected to be incorporated into the products of our industrial partner.

• [2018-2] B. Sonkoly, M. Szabó, B. Németh, A. Majdán, G. Pongrácz, L. Toka, “FERO: Fast and Efficient Resource Orchestrator for a Data Plane Built on Docker and DPDK,” In Proc. of IEEE INFOCOM 2018

• [2018-3] B. Németh, M. Szalay, J. Doka, M. Rost, S. Schmid, L. Toka, B. Sonkoly, “Fast and efficient network service embedding method with adaptive offloading to the edge,” In Proc. of IEEE INFOCOM (Workshops) 2018

• [2018-4] D. Haja, M. Szabo, M. Szalay, A. Nagy, A. Kern, L. Toka, B. Sonkoly, “How to orchestrate a distributed OpenStack,” In Proc. of IEEE INFOCOM (Workshops) 2018

• [2018-5] M. Cserep, A. Recse, R. Szabo, L. Toka, “Business network formation among 5G providers,” In Proc. of IEEE INFOCOM (Workshops) 2018

• [2018-6] J. Harmatos, D. Jocha, R. Szabó, B. P. Gero, B. Németh, J. Czentye, B. Sonkoly, “Self-healing in multi-constrained multi-actor virtualization orchestration,” In Proc. of IEEE INFOCOM 2018 (Demo)