{"id":10337,"date":"2023-06-05T16:18:25","date_gmt":"2023-06-05T14:18:25","guid":{"rendered":"https:\/\/www.iese.fraunhofer.de\/blog\/?p=10337"},"modified":"2025-01-06T12:02:38","modified_gmt":"2025-01-06T11:02:38","slug":"performance-efficiency","status":"publish","type":"post","link":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/","title":{"rendered":"Measuring the Performance Efficiency of Eclipse BaSyx"},"content":{"rendered":"<p class=\"lead\">Industrie 4.0 (I4.0) is about end-to-end digitalization between IT (information technology) and OT (operation technology), i.e., the shopfloor. For example, the production of small lot sizes requires the ability to efficiently change production. Modern approaches, such as service-oriented manufacturing and <a href=\"https:\/\/www.iese.fraunhofer.de\/de\/leistungen\/digitaler-zwilling.html\" target=\"_blank\" rel=\"noopener\">Digital Twins<\/a>, support this. With this technology, process changes can be implemented much faster than with traditional automation. In order to support realistic deployments, an Industrie 4.0 middleware needs to support a large number of Digital Twins. In this post, our experts from <a href=\"https:\/\/www.iese.fraunhofer.de\/\" target=\"_blank\" rel=\"noopener\">Fraunhofer IESE<\/a> provide an overview of the performance efficiency of <a href=\"https:\/\/www.eclipse.org\/basyx\/\" target=\"_blank\" rel=\"noopener\">Eclipse BaSyx<\/a>, an open-source Industrie 4.0 middleware that enables the realization of Digital Twins based on <a href=\"https:\/\/www.iese.fraunhofer.de\/blog\/industry-4-0-it-infrastructure-for-digital-twins\/\" target=\"_blank\" rel=\"noopener\">Asset Administration Shells<\/a>.<\/p>\n<p><strong>Performance Efficiency<\/strong> is one of the key quality attributes to be considered when designing an Industrie 4.0 solution. The <a href=\"https:\/\/iso25000.com\/index.php\/en\/iso-25000-standards\/iso-25010?limit=3\" target=\"_blank\" rel=\"noopener\">ISO 25010<\/a> [1] standard defines <strong>Performance Efficiency<\/strong> as <em>\u201cthe performance relative to the amount of resources used under stated conditions\u201d.<\/em> It is composes of the following sub-characteristics:<\/p>\n<ul>\n<li><strong>Time behavior<\/strong>: Degree to which the response, processing times and throughput rates of a product or system, when performing its functions, meet requirements.<\/li>\n<li><strong>Resource utilization<\/strong>: Degree to which the amounts and types of resources used by a product or system, when performing its functions, meet requirements.<\/li>\n<li><strong>Capacity<\/strong>: Degree to which the maximum limits of a product or system parameter meet requirements.<\/li>\n<\/ul>\n<p>In this post, we will describe the results of the <strong>time behavior<\/strong> of Digital Twins implemented with Eclipse BaSyx. Monitoring of the <strong>resource utilization<\/strong> also took place, but does not need a major focus for now.<\/p>\n<h2><strong>The Asset Administration Shell in a nutshell<\/strong><\/h2>\n<p>The Asset Administration Shell (AAS) is a digital representation of an asset. It has one or multiple IDs that describe the asset. Every AAS should have at least one worldwide IDs that uniquely identify the asset. In addition, AASs may represent additional IDs that are unique in their context, e.g., a serial number. An AAS consists of multiple submodels that describe the represented real-world asset from the viewpoint of different use cases.<\/p>\n<p>The submodels therefore focus on specific kinds of information: For example. one AAS submodel defines a digital nameplate for the asset, another submodel provides documentation, a third submodel describes the current status of a resource and connects, for example, to a device to obtain this information. Depending on the kind of asset that is represented by the AAS, and depending on the environment, an AAS may provide different sets of AAS submodels. AAS submodels may be standardized or not. Important submodels that are imperative for realizing digital supply chains are being standardized under the umbrella of the IDTA. Other, possibly more specific submodels are probably not covered by a standard, but stakeholders have agreed on them.<\/p>\n<h2><strong>About Eclipse BaSyx<\/strong><\/h2>\n<p>Eclipse BaSyx is an Industrie 4.0 middleware that implements the infrastructure for AASs and provides the necessary tools for creating AASs. Furthermore, it offers tools for the import of AAS types and instances, and for the dynamic instantiation of AASs. In addition, Eclipse BaSyx enables active AASs, which yield a behavior to autonomously collect and aggregate data, or to implement harmonized interfaces to devices by acting as device drivers. In this way, for example, characteristic curves do not need to be implemented in the process logically anymore (and therefore do not need to be changed when the device type changes).<\/p>\n<p>Eclipse BaSyx implements three main AAS components: the AAS Server, a repository for hosting AAS; the Registry, which locates AASs on the AAS server(s) based on the AAS ID, and the Data Bridge, which connects data sources to AAS properties. Furthermore, Eclipse BaSyx provides SDKs for Java, C#, and C++ that allow implementers to create AAS-enabled applications.<\/p>\n<p>To ensure scalability, Eclipse BaSyx stores all information in an external information provider. For the following benchmarks, we selected MongoDB, an NoSQL database. Besides, BaSyx supports backends such as SQL, standard HDD, memory (for testing), and S3 storage. The communication between users, other software systems, and AAS Servers and Registries can be implemented using different technologies, like http-rest interfaces, MQTT, OPCUA, and many more.<\/p>\n<h2><strong>What is being tested by Performance Tests<\/strong><\/h2>\n<p>For measuring the performance efficiency of Eclipse BaSyx, we tested the following:<\/p>\n<ol>\n<li><em>How long does it take between sending an http-request to the AAS-Server and receiving an answer back, i.e., <\/em>what is the <strong>response time of the system<\/strong>? This response time depends on the constraints of the software and the hardware on which BaSyx is running.<\/li>\n<li><em>Implicit effects of the network, i.e<\/em>., delay of requests. As an infrastructure for BaSyx, our experts set up a Kubernetes cluster to ensure that the parameters of a network with industrial complexity can be tested.<\/li>\n<\/ol>\n<h2><strong>Test scenarios for testing the Performance Efficiency of Eclipse BaSyx<\/strong><\/h2>\n<p>Since it is not enough to perform only one benchmark to make a statement about the system, we defined a variety of scenarios. In every one of them, we adapted several parameters to mimic data flows in a real production system and determined the following parameters for our testing:<\/p>\n<ol>\n<li><strong>Number of clients or virtual users (VU):<\/strong> Real production environments consist of a large number of independent virtual users. Our tests ran with 10 &#8211; 30,000 VU\u2019s simultaneously.<\/li>\n<li><strong>Frequency of VU&#8217;s: <\/strong>Requests are sent at different frequencies without a predictable order. To cover this, we varied the frequency requests sent by the VU\u2019s from 1 \u2013 100 Hz. We additionally benchmarked VUs without frequency restrictions.<\/li>\n<li><strong>Hardware: <\/strong>We executed our benchmarks on a Kubernetes system, which will be described in more detail in the next section.\n<ol>\n<li>We varied the <strong>virtualization (configuration) of the virtual machines (VM\u2019s)<\/strong> as well as the<\/li>\n<li><strong>Physical hardware<\/strong> to determine the dependency of the software on the available resources<\/li>\n<\/ol>\n<\/li>\n<li><strong>Number of AASs present in the system: <\/strong>The larger and more complex a system becomes, the more AASs the middleware has to process. In this regard, we tested the Eclipse BaSyx software with 200 \u2013 8000 active AASs plus submodels (SM).<\/li>\n<li><strong>Kind of request<\/strong>: We tested\n<ol>\n<li><strong>Read requests<\/strong> and<\/li>\n<li><strong>Write requests <\/strong>to check for any performance differences. For the same reason, we tested the influence of<\/li>\n<\/ol>\n<\/li>\n<li><strong>Randomness <\/strong> More precisely, the difference between:\n<ol>\n<li><strong>All requests target the same AAS<\/strong> and<\/li>\n<li><strong>Every request targets a random AAS<\/strong><\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<h2><strong>Defining the test scenarios<\/strong><\/h2>\n<p>Based on these testing requirements, we defined a test order for all possible combinations of these parameters, as shown in <strong>Figure 1<\/strong>.<\/p>\n<figure id=\"attachment_10341\" aria-describedby=\"caption-attachment-10341\" style=\"width: 698px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-10341 size-large\" src=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Test-order-698x540.png\" alt=\"Explaining the order of the tests for measuring performance efficiency.\" width=\"698\" height=\"540\" srcset=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Test-order-698x540.png 698w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Test-order-400x309.png 400w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Test-order-768x594.png 768w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Test-order.png 1049w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption id=\"caption-attachment-10341\" class=\"wp-caption-text\">Figure 1 &#8211; Test order<\/figcaption><\/figure>\n<p>First of all, we defined the hardware setups. For this purpose, we used the same physical hardware for all benchmarks, but configured different virtualizations, i.e., with a varying numbers of nodes (VMs) present in the cluster and different amounts of computational resources. We started with one node and four CPU cores + 16 GB RAM and a second cluster with four nodes and four CPU cores + 16 GB RAM per node, as depicted in <strong>Figure 2<\/strong>. On each of these hardware configurations, we executed multiple test series with different numbers of AASs. Regarding the application behavior, we tested read and write operations, and we targeted either the same AAS or a random one using Eclipse BaSyx version 1.2.<\/p>\n<figure id=\"attachment_10338\" aria-describedby=\"caption-attachment-10338\" style=\"width: 698px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-10338 size-large\" src=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Cluster-configurations-698x338.png\" alt=\"Showing the difference between the used cluster configurations\" width=\"698\" height=\"338\" srcset=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Cluster-configurations-698x338.png 698w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Cluster-configurations-400x194.png 400w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Cluster-configurations-768x372.png 768w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Cluster-configurations-1320x639.png 1320w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Cluster-configurations.png 1335w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption id=\"caption-attachment-10338\" class=\"wp-caption-text\">Figure 2 &#8211; Cluster configurations<\/figcaption><\/figure>\n<h2><strong>Performance test setup<\/strong><\/h2>\n<p>For testing the performance of Eclipse BaSyx, we chose Kubernetes. It allows hosting code within container software on VMs (virtual machines), called nodes. The infrastructure for the Kubernetes clusters is hosted by our partner <a href=\"https:\/\/www.netapp.com\/blog\/industry-4-0-infrastructure-for-digital-twins\/\">NetApp<\/a>, which also uses Digital Twins and BaSyx.<\/p>\n<p>Kubernetes offers:<\/p>\n<ol>\n<li>Automated health and readiness checks<\/li>\n<li>Automated networking between software components<\/li>\n<li>Constant and defined distribution of computational resources across containers<\/li>\n<li>Monitoring of the resource consumption of the containers<\/li>\n<\/ol>\n<p>To eliminate network delays to the greatest possible extent, we decided to host the testing software in another Kubernetes cluster in the same data center (cf. <strong>Figure 3<\/strong>). The connection speed between the two clusters was &gt; 10 GB\/s. Another reason for using a second Kubernetes cluster for the testing software was its own resource consumption. Kubernetes ensures that the executed software always has sufficient computational resources and provides an intelligent resource distribution, as long as technically possible. This minimizes the impact of the testing infrastructure on the test results.<\/p>\n<figure id=\"attachment_10339\" aria-describedby=\"caption-attachment-10339\" style=\"width: 698px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-10339 size-large\" src=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/General-setup-698x418.png\" alt=\"Showing the simplified test setup for the measurement of performance efficiency.\" width=\"698\" height=\"418\" srcset=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/General-setup-698x418.png 698w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/General-setup-400x239.png 400w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/General-setup-768x460.png 768w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/General-setup.png 1240w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption id=\"caption-attachment-10339\" class=\"wp-caption-text\">Figure 3 &#8211; General setup<\/figcaption><\/figure>\n<p>After defining the general setup, the testing system itself had to be designed. As previously mentioned, our goal was to simulate the behavior of a real factory environment. Therefore, we defined the number of VUs that communicate with BaSyx components over the http-rest interface, as shown in <strong>Figure 4<\/strong>. In doing so, we also measured and saved the duration of each request. We used K6, an open-source project, for loading testing applications and for orchestrating the testing environment. A K6 configuration defines the Kubernetes resources, for example the number of VUs and the simulated applications for a benchmarking scenario.<\/p>\n<p>In our case, the duration of each request by each VU was measured and saved to an InfluxDB. We visualized the results after each benchmark with Eclipse Grafana. The resource consumption of each container, the so-called metrics, were monitored through <strong>Rafay<\/strong>, a management software\/platform for Kubernetes clusters.<\/p>\n<figure id=\"attachment_10340\" aria-describedby=\"caption-attachment-10340\" style=\"width: 698px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-10340 size-large\" src=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Setup-for-testing-698x403.png\" alt=\"Showing the technical test setup as an overview\" width=\"698\" height=\"403\" srcset=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Setup-for-testing-698x403.png 698w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Setup-for-testing-400x231.png 400w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Setup-for-testing-768x443.png 768w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/Setup-for-testing.png 874w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption id=\"caption-attachment-10340\" class=\"wp-caption-text\">Figure 4 &#8211; Setup for testing<\/figcaption><\/figure>\n<h2><strong>Performance Efficieny of Eclipse BaSyx &#8211; results<\/strong><\/h2>\n<p>In <strong>Figures 5 &#8211; 7 <\/strong>below, the Grafana dashboard depicts a test series with 10-1000 VUs with a frequency of 10 requests per second per VU. The Eclipse BaSyx middleware was hosted on a cluster with four nodes, with four CPU cores and 16 GB RAM for each node. Each test lasted for five minutes, using write requests to query a random AAS.<\/p>\n<figure id=\"attachment_10351\" aria-describedby=\"caption-attachment-10351\" style=\"width: 698px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-10351\" src=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record1a-698x365.png\" alt=\"Showing an example of the test results\" width=\"698\" height=\"365\" srcset=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record1a-698x365.png 698w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record1a-400x209.png 400w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record1a-768x402.png 768w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record1a.png 1170w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption id=\"caption-attachment-10351\" class=\"wp-caption-text\">Figure 5 &#8211; Visualization of test results &#8211; 1<\/figcaption><\/figure>\n<figure id=\"attachment_10352\" aria-describedby=\"caption-attachment-10352\" style=\"width: 698px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-10352\" src=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record2a-698x361.png\" alt=\"Showing an example of the test results\" width=\"698\" height=\"361\" srcset=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record2a-698x361.png 698w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record2a-400x207.png 400w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record2a-768x397.png 768w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record2a.png 1176w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption id=\"caption-attachment-10352\" class=\"wp-caption-text\">Figure 6 &#8211; Visualization of test results &#8211; 2<\/figcaption><\/figure>\n<figure id=\"attachment_10353\" aria-describedby=\"caption-attachment-10353\" style=\"width: 698px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-large wp-image-10353\" src=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record3a-698x353.png\" alt=\"Showing an example of the test results\" width=\"698\" height=\"353\" srcset=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record3a-698x353.png 698w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record3a-400x202.png 400w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record3a-768x388.png 768w, https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/test-record3a.png 1213w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption id=\"caption-attachment-10353\" class=\"wp-caption-text\">Figure 7 &#8211; Visualization of test results &#8211; 3<\/figcaption><\/figure>\n<p>Then we conducted a review for every test series. This led to the determination and verification of the following observations:<\/p>\n<ol>\n<li>It makes <strong>no difference<\/strong> for the duration of the request if <strong>only one AAS<\/strong> is targeted with all VUs or if each request targets a <strong>random AAS<\/strong>.<\/li>\n<li>It makes almost <strong>no difference<\/strong> for the duration of the request duration whether a <strong>write or read request was performed<\/strong>.<\/li>\n<li>Regarding the duration of the request, <strong>the number of AASs made no difference<\/strong>.<\/li>\n<li><strong>Quadrupling<\/strong> the <strong>computational resources<\/strong> yielded a <strong>quadrupling of the number of processed requests<\/strong>, which indicates <strong>linear growth.<\/strong><\/li>\n<li>The <strong>minimum duration of a request was 2 ms<\/strong> due to network limitations.<\/li>\n<li>With the <strong>hardware<\/strong> <strong>configuration used<\/strong>, Eclipse BaSyx responded to 1000 VUs (with 10 requests per second per VU, so <strong>10,000 request\/s<\/strong>) <strong>within 60 ms<\/strong> on average<\/li>\n<li>The RAM consumption of the AAS Server was influenced by the number of AASs targeted by requests. If the <strong>number of AASs was increased by a factor of 40<\/strong>, the <strong>RAM <\/strong><strong>consumption increased by a factor of 1.6<\/strong>.<\/li>\n<li>The <strong>number of AASs did not affect the CPU consumption<\/strong> of the AAS Server<\/li>\n<\/ol>\n<h2><strong>Conclusion<\/strong><\/h2>\n<p>In a nutshell, we conclude that Eclipse BaSyx is able to handle over 10,000 requests per second, with little hardware consumption. In addition, the growth rate of the hardware consumption related to the growth rate of the number of AASs present in the system also looks positive. More specifically, every AAS in the system was found to be actively used and therefore present in the RAM of the AAS Server. In a real industrial setup, a subset of AASs would be actively used and the rest would be stored in the data backend until they are needed.<\/p>\n<p>Despite these positive results, the resource consumption related to AASs in use is a potential bottleneck. This information will be considered for future development processes with regard to Eclipse BaSyx.<\/p>\n<p>Tests with bigger clusters, more computational resources, and better hardware will be able to show the full potential of Eclipse BaSyx.<\/p>\n<p>&nbsp;<\/p>\n<div class=\"info-box\">\n<p><strong>You want to find out more about Eclipse BaSyx and Digital Twins?<\/strong><\/p>\n<p>&nbsp;<\/p>\n<p>Then reach out to our expert <a href=\"mailto:frank.schnicke@iese.fraunhofer.de\" target=\"_blank\" rel=\"noopener\">Frank Schnicke<\/a> and\/or read more in one of the following articles on our Fraunhofer IESE Blog or on our websites:<\/p>\n<p>&nbsp;<\/p>\n<ul>\n<li><a href=\"https:\/\/www.iese.fraunhofer.de\/en\/customers_industries\/digitalisierung-produktion\/industrie40.html\" target=\"_blank\" rel=\"noopener\">Fraunhofer IESE website on &#8222;Industrie 4.0&#8220;<\/a><\/li>\n<li><a href=\"https:\/\/www.iese.fraunhofer.de\/content\/dam\/iese\/en\/dokumente\/Innovition-Themes\/DigitalerZwilling_AR2017.pdf\" target=\"_blank\" rel=\"noopener\">Fraunhofer IESE interview about &#8222;Digital Twins&#8220;<\/a><\/li>\n<li><a href=\"https:\/\/www.iese.fraunhofer.de\/blog\/industrie-4-0-it-infrastructure-for-digital-twins-part2\/\" target=\"_blank\" rel=\"noopener\">Fraunhofer IESE blog post on &#8222;Implementing the Industrie 4.0 IT Infrastructure for Digital Twins\u201c&#8220;<\/a><\/li>\n<li><a href=\"https:\/\/www.iese.fraunhofer.de\/blog\/continuous-engineering-industrie-40\/\" target=\"_blank\" rel=\"noopener\">Fraunhofer IESE blog post on &#8222;Continuous Engineering for Industrie 4.0&#8220;<\/a><\/li>\n<\/ul>\n<\/div>\n<p>&nbsp;<\/p>\n<p>[1] ISO 25000 Standard. &#8222;ISO\/IEC 25010 Systems and software engineering \u2014 Systems and software Quality Requirements and Evaluation (SQuaRE) \u2014 System and software quality models.&#8220; ISO\/IEC, (2017).<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Industrie 4.0 (I4.0) is about end-to-end digitalization between IT (information technology) and OT (operation technology), i.e., the shopfloor. For example, the production of small lot sizes requires the ability to efficiently change production. Modern approaches, such as service-oriented manufacturing and&#8230;<\/p>\n","protected":false},"author":123,"featured_media":10405,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[94],"tags":[537,120,48,198,477],"coauthors":[535,317],"class_list":["post-10337","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-industrie-4-0","tag-basyx","tag-digitaler-zwilling","tag-digitalisierung","tag-english","tag-virtual-engineering"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Measuring the Performance Efficiency of Eclipse BaSyx - Blog des Fraunhofer IESE<\/title>\n<meta name=\"description\" content=\"In this blog post, we want to show, how Performance Tests help to identify the Performance Efficiency of the middleware Eclipse BaSyx.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Measuring the Performance Efficiency of Eclipse BaSyx - Blog des Fraunhofer IESE\" \/>\n<meta property=\"og:description\" content=\"In this blog post, we want to show, how Performance Tests help to identify the Performance Efficiency of the middleware Eclipse BaSyx.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/\" \/>\n<meta property=\"og:site_name\" content=\"Fraunhofer IESE\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/FraunhoferIESE\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-06-05T14:18:25+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-01-06T11:02:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/basyx_blog.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1400\" \/>\n\t<meta property=\"og:image:height\" content=\"702\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Marius Zunkel, Frank Schnicke\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@FraunhoferIESE\" \/>\n<meta name=\"twitter:site\" content=\"@FraunhoferIESE\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"Marius Zunkel\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"10\u00a0Minuten\" \/>\n\t<meta name=\"twitter:label3\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data3\" content=\"Marius Zunkel, Frank Schnicke\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/\"},\"author\":{\"name\":\"Marius Zunkel\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#\\\/schema\\\/person\\\/da4ed0be079a8824bb0d6c2e8be70cfb\"},\"headline\":\"Measuring the Performance Efficiency of Eclipse BaSyx\",\"datePublished\":\"2023-06-05T14:18:25+00:00\",\"dateModified\":\"2025-01-06T11:02:38+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/\"},\"wordCount\":2060,\"publisher\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/05\\\/basyx_blog.jpg\",\"keywords\":[\"BaSyx\",\"Digitaler Zwilling\",\"Digitalisierung\",\"English\",\"Virtual Engineering\"],\"articleSection\":[\"Industrie 4.0\"],\"inLanguage\":\"de\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/\",\"url\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/\",\"name\":\"Measuring the Performance Efficiency of Eclipse BaSyx - Blog des Fraunhofer IESE\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/05\\\/basyx_blog.jpg\",\"datePublished\":\"2023-06-05T14:18:25+00:00\",\"dateModified\":\"2025-01-06T11:02:38+00:00\",\"description\":\"In this blog post, we want to show, how Performance Tests help to identify the Performance Efficiency of the middleware Eclipse BaSyx.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/05\\\/basyx_blog.jpg\",\"contentUrl\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/05\\\/basyx_blog.jpg\",\"width\":1400,\"height\":702,\"caption\":\"Performance Efficiency of Eclipse BaSyx\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/performance-efficiency\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Startseite\",\"item\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Measuring the Performance Efficiency of Eclipse BaSyx\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/\",\"name\":\"Fraunhofer IESE\",\"description\":\"Blog des Fraunhofer-Institut f\u00fcr Experimentelles Software Engineering\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#organization\",\"name\":\"Fraunhofer IESE\",\"url\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2016\\\/08\\\/fhg_iese_logo.png\",\"contentUrl\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2016\\\/08\\\/fhg_iese_logo.png\",\"width\":183,\"height\":50,\"caption\":\"Fraunhofer IESE\"},\"image\":{\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/FraunhoferIESE\\\/\",\"https:\\\/\\\/x.com\\\/FraunhoferIESE\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/fraunhoferiese\\\/\",\"https:\\\/\\\/www.youtube.com\\\/c\\\/FraunhoferIESE\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/#\\\/schema\\\/person\\\/da4ed0be079a8824bb0d6c2e8be70cfb\",\"name\":\"Marius Zunkel\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/03\\\/becker_C97A8401_web8-96x96.jpg24fb411ef1695d922a04c43fa9b11e71\",\"url\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/03\\\/becker_C97A8401_web8-96x96.jpg\",\"contentUrl\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/wp-content\\\/uploads\\\/2023\\\/03\\\/becker_C97A8401_web8-96x96.jpg\",\"caption\":\"Marius Zunkel\"},\"url\":\"https:\\\/\\\/www.iese.fraunhofer.de\\\/blog\\\/author\\\/marius-becker\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Measuring the Performance Efficiency of Eclipse BaSyx - Blog des Fraunhofer IESE","description":"In this blog post, we want to show, how Performance Tests help to identify the Performance Efficiency of the middleware Eclipse BaSyx.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/","og_locale":"de_DE","og_type":"article","og_title":"Measuring the Performance Efficiency of Eclipse BaSyx - Blog des Fraunhofer IESE","og_description":"In this blog post, we want to show, how Performance Tests help to identify the Performance Efficiency of the middleware Eclipse BaSyx.","og_url":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/","og_site_name":"Fraunhofer IESE","article_publisher":"https:\/\/www.facebook.com\/FraunhoferIESE\/","article_published_time":"2023-06-05T14:18:25+00:00","article_modified_time":"2025-01-06T11:02:38+00:00","og_image":[{"width":1400,"height":702,"url":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/basyx_blog.jpg","type":"image\/jpeg"}],"author":"Marius Zunkel, Frank Schnicke","twitter_card":"summary_large_image","twitter_creator":"@FraunhoferIESE","twitter_site":"@FraunhoferIESE","twitter_misc":{"Verfasst von":"Marius Zunkel","Gesch\u00e4tzte Lesezeit":"10\u00a0Minuten","Written by":"Marius Zunkel, Frank Schnicke"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/#article","isPartOf":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/"},"author":{"name":"Marius Zunkel","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#\/schema\/person\/da4ed0be079a8824bb0d6c2e8be70cfb"},"headline":"Measuring the Performance Efficiency of Eclipse BaSyx","datePublished":"2023-06-05T14:18:25+00:00","dateModified":"2025-01-06T11:02:38+00:00","mainEntityOfPage":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/"},"wordCount":2060,"publisher":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#organization"},"image":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/basyx_blog.jpg","keywords":["BaSyx","Digitaler Zwilling","Digitalisierung","English","Virtual Engineering"],"articleSection":["Industrie 4.0"],"inLanguage":"de"},{"@type":"WebPage","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/","url":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/","name":"Measuring the Performance Efficiency of Eclipse BaSyx - Blog des Fraunhofer IESE","isPartOf":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/#primaryimage"},"image":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/#primaryimage"},"thumbnailUrl":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/basyx_blog.jpg","datePublished":"2023-06-05T14:18:25+00:00","dateModified":"2025-01-06T11:02:38+00:00","description":"In this blog post, we want to show, how Performance Tests help to identify the Performance Efficiency of the middleware Eclipse BaSyx.","breadcrumb":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/#primaryimage","url":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/basyx_blog.jpg","contentUrl":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/basyx_blog.jpg","width":1400,"height":702,"caption":"Performance Efficiency of Eclipse BaSyx"},{"@type":"BreadcrumbList","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/performance-efficiency\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Startseite","item":"https:\/\/www.iese.fraunhofer.de\/blog\/"},{"@type":"ListItem","position":2,"name":"Measuring the Performance Efficiency of Eclipse BaSyx"}]},{"@type":"WebSite","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#website","url":"https:\/\/www.iese.fraunhofer.de\/blog\/","name":"Fraunhofer IESE","description":"Blog des Fraunhofer-Institut f\u00fcr Experimentelles Software Engineering","publisher":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.iese.fraunhofer.de\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#organization","name":"Fraunhofer IESE","url":"https:\/\/www.iese.fraunhofer.de\/blog\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2016\/08\/fhg_iese_logo.png","contentUrl":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2016\/08\/fhg_iese_logo.png","width":183,"height":50,"caption":"Fraunhofer IESE"},"image":{"@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/FraunhoferIESE\/","https:\/\/x.com\/FraunhoferIESE","https:\/\/www.linkedin.com\/company\/fraunhoferiese\/","https:\/\/www.youtube.com\/c\/FraunhoferIESE"]},{"@type":"Person","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/#\/schema\/person\/da4ed0be079a8824bb0d6c2e8be70cfb","name":"Marius Zunkel","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/03\/becker_C97A8401_web8-96x96.jpg24fb411ef1695d922a04c43fa9b11e71","url":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/03\/becker_C97A8401_web8-96x96.jpg","contentUrl":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/03\/becker_C97A8401_web8-96x96.jpg","caption":"Marius Zunkel"},"url":"https:\/\/www.iese.fraunhofer.de\/blog\/author\/marius-becker\/"}]}},"jetpack_featured_media_url":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-content\/uploads\/2023\/05\/basyx_blog.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/posts\/10337","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/users\/123"}],"replies":[{"embeddable":true,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/comments?post=10337"}],"version-history":[{"count":20,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/posts\/10337\/revisions"}],"predecessor-version":[{"id":10468,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/posts\/10337\/revisions\/10468"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/media\/10405"}],"wp:attachment":[{"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/media?parent=10337"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/categories?post=10337"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/tags?post=10337"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.iese.fraunhofer.de\/blog\/wp-json\/wp\/v2\/coauthors?post=10337"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}