Multi-Agent System Development and Application for Supply Chain Management Tasks

– In today’s dynamically changing environment, we need to be able to respond in a timely manner to changes in supply chain processes. Software agents are successfully used in supply chain management tasks for a variety of purposes. The behaviour of agents is determined by the purpose of their development, and the effectiveness of the use of agents is considered in accordance with the purpose of their development. The paper presents research on the development of a multi-agent system for supply chain management, focusing on the steps of developing a multi-agent system. The choice of each algorithm for agents is analysed and argued. The application of the developed multi-agent system for supply chain management is also described in the paper. The efficiency of application of the developed multi-agent system is presented as well.


I. INTRODUCTION
In today's dynamically changing environment, we need to be able to respond in a timely manner to changes in supply chain processes. Software agents are successfully used in supply chain management tasks for a variety of purposes. The behaviour of agents is determined by the purpose of their development, and the effectiveness of the use of agents is considered in accordance with the purpose of their development.
Despite the availability of multi-agent systems the problem of using agents to improve the efficiency of supply chain management is still relevant. The system to be developed must be adaptable to different applications and be able to complement existing systems.
There are a variety of supply chain performance improvement technologies, such as information technology, business management, artificial intelligence, or simulation modelling software.
In order to improve supply chain management performance, researchers were divided into four groups: • the first group studied the facility location task; • the second group suggested the use of simulation modelling software; • the third group used the LEAN production process; • the fourth group of authors suggested the use of agents for supply chain management. * Corresponding author. E-mail: darja.plinere@rtu.lv All of these researchers have achieved their goals, and these research directions can be successfully applied, according to a specific task: 1. Facility location is a task of strategic planning; it is mainly necessary at the strategic planning stage of the supply chain. It addresses the following issues: where to build or lease a factory, warehouse and determine their quantity for the optimal operation of the company, how to determine the location of equipment and their capacity. In case of an existing supply chain, the challenge could be to open the next building or buy equipment to meet greater customer demand. 2. The advantage of using a simulation modelling tool is to apply different strategies and determine the best strategy or better configuration of the supply chain. 3. The LEAN methodology can ensure continuous operational improvement by eliminating redundant and non-value-adding activities in the supply chain. 4. A multi-agent system can offer improved supply chain management by allocating tasks to agents, and they will work together to achieve better system performance results [1].
Analysing the above research directions, it has been concluded that multi-agent systems are the most suitable ones for improving supply chain management performance, as they are able to offer continuous improvement in dynamically changing conditions, include other improvement methods and be successfully applied in both existing and new companies.
The paper is organised as follows: next section presents multi-agent system development steps, the third section discusses agent communication means, the fourth explains algorithms of developed agents in the multi-agent system, the fifth section summarises the application of the developed multiagent system, the sixth section presents performance measurement and its indicators and the paper ends with conclusions and ideas for further work.

II. MULTI-AGENT SYSTEM DEVELOPMENT
The authors propose summarising several definitions [2], [3], [4] and considering that the definition of an agent is understood as follows: an agent is an autonomous and proactive computing system that interacts with its environment, responds in a timely manner to environmental changes and can communicate and collaborate with other agents or users to achieve their goal.
Multi-agent systems are systems that consist of a number of interactive computing elements known as agents.
The advantages of using a multi-agent system are as follows: agents are dynamic, autonomous, are able to interact with other agents and/or users and databases and servers, able to adapt to new conditions, able to learn, be proactive, can be mobile, independent, intellectual, goal-oriented, reactive and multiagent systems that can split processes so that they run in parallel.
The authors have taken two classifications of agents as the basis for developing the agent architecture [2], [4]: • The agent classification includes eight agent types [2]: collaboration, interface, mobile, information, reactive, hybrid, smart and heterogeneous agent systems. • Agent architecture can be classified by dividing agents into simple reflexive, model-based, model-and goalbased, utility-based, and learning agents [4].
A heterogeneous system of agents and model-and goalbased agents have been chosen.
The number of agents affects the efficiency of the whole system; a larger number of agents, on the one hand, saves time through synchronized performance, but on the other hand requires additional time for communication. However, a larger number of agents contributes to the generality of the system architecture, which means that the system can be easily reconfigured for a specific company.
It is proposed to divide supply chain management functions into four agents: Seller agent, Procurement agent, Assembling agent and Inventory management agent. These functions exist at each node in the supply chain, so each node can be represented by similar agents.
In order to include the communication function between the supply chain nodes, it is proposed to create two more Supplier and Distributor / Customer company agents -the Supplier Agent and the Customer Agent. The aims and behaviours of these two agents are similar to the manufacturer's Seller agent and Procurement agent, respectively.

III. AGENT COMMUNICATION
The key feature of an agent is its ability to interact with its environment. The agent perceives its environment, uses what is perceived to select an action, and then performs the action through its effectors [5].
In case of a multi-agent system, agents may need to interact with each other, and these types of interactions are as follows: • cooperation -action to achieve a common goal; • coordination -organising problem-solving activities to avoid harmful interactions or to use positive interactions; • negotiation -reaching an agreement that is acceptable to all parties involved.
Agent interaction requires a type of communication. The authors' analysis [1] of existing multi-agent systems has revealed that existing multi-agent systems use a blackboard system or ontology for agent communication.
Multi-agent systems are suitable for areas where there is an interaction between different people or organisations with different and potentially conflicting goals and secret information. A supply chain is considered to be a set of intelligent agents, where each agent is responsible for one or more activities in the supply chain. The ontology, in its turn, describes the subject area and becomes a mechanism to help understand and analyse the flow of information between agents. Using an ontology for a multi-agent system provides these advantages: an ontology allows knowledge to be structured and shared, increases the reliability of the multi-agent system, and provides a basis for interaction between agents [1].
Communication between agents in a multi-agent system can be achieved using a blackboard architecture. They do not communicate with each other, but use the blackboard as a central depository, place the data there, and the agents wait for the appropriate data to contribute. Agents work consistently; the blackboard is used as a central repository for all shared information. All the partial results of the agents are posted on the blackboard and used by other agents as soon as the final solution is found [5].
The authors believe that the use of ontology for agent communication is the most promising way of communication, and proposes to introduce information sharing in a multi-agent system similar to the blackboard architecture.

IV. AGENT ALGORITHMS
It is proposed to develop four agents [6] that cover all the supply chain management functions of the manufacturer: • The Procurement agent is responsible for the selection of suppliers and communication with them. • The Assembling agent is engaged in production planning and re-planning. • Inventory management agent deals with demand forecasting, inventory management and replenishment.
• The Seller agent is responsible for receiving customer orders.

A. Seller Agent Development
The goal of the Seller agent is to provide a high level of service (customer satisfaction) and to ship the goods immediately.
The task of the Seller agent is to receive orders from buyers and immediately send the ordered product (s) with a high level of service. There is a return mechanism in the supply chain if the buyer has a defective product or is dissatisfied with the purchased product. The Seller agent ensures the return process to the manufacturer.
It is assumed that the Seller agent has no information or it has limited information on future customer demand, although the exchange of information on demand can benefit all participants in the supply chain.
The following behaviour of the Seller agent is offered in the developed multi-agent system: the agent is responsible for receiving orders from customers, it interacts with buyers for delivery time. The price of the product is fixed, the agent cannot change it.
Initially, the Seller agent is idle; it is activated when a new order is received from the Customer agent. The rules of Seller agent behaviour are as follows: • If the ordered quantity is available in stock, reserve it and offer the Customer agent the price and delivery time.
• If the Customer agent accepts the price and delivery time, then start the sales procedure. Communication between the Seller agent and the Customer agent is performed using the ManufacturerCustomer ontology. These agents have a common understanding of the information they are talking about.

B. Assembling Agent Development
The goal of the Assembling agent is to organise the production process in such a way as to reduce production time while maintaining or increasing the level of product quality.
Assembling agent is engaged in the production or assembly of products from raw materials, that is, the assembly of finished products from raw materials in accordance with the production technology and their transfer to the warehouse of finished goods. It has no knowledge of future sales of finished products, as well as no forecast for the next order.
The main tasks of the Assembling agent are production planning and improvement of production processes. Improving production processes, in turn, includes the following subtasks: efficient equipment load, reduction of production time and organisation of a continuous flow of products in the production / assembly department.
The developed multi-agent system offers the following behaviour of the Assembling agent: the agent is responsible for production planning and re-planning, existing production orders, as well as plans the execution of new production orders and re-plans the production sequence, if it is possible to reduce production time.
The Assembling agent behaviour algorithm is as follows: • For n existing products, the different possible variants are n! If it is necessary to produce five types of products, then n! = 5!; • For each of n! variants, total production time must be calculated. A well-thought-out production sequence has the lowest total production time; • If the total production time exceeds the deadline, the production process priority level based on the results of the ABC analysis is used: the highest priority is for class A products, then for class B products and the lowest priority is for class C products. If the products belong to the same class, then the priority of the production process is given to the product with a shorter production time.
This behaviour algorithm ensures efficient equipment load and reduces total production time, thus reducing production costs and eliminating unnecessary waiting time.

C. Inventory Management Agent Development
From a material management point of view [6], stocks are defined as "a usable but idle resource with a certain economic value". Physical stocks must be available to meet demand, otherwise it will lead to production delays, shortages and / or customer dissatisfaction. However, stockpiling is not free, so the paradox is that stocks are needed but not desirable. This situation makes inventory management a difficult problem in supply chain management.
In order to meet customer demand, stocks must be available for at least the following reasons: the lead time between ordering and delivery is quite long, but at the same time customer demand must be met if there is uncertainty in demand or seasonal demand, and so on. In order to reduce stocks and their storage costs, inventory management is proposed [7].
The tasks of inventory management are to analyse stock balances, sales by customer groups, sales in the previous period and seasonality in the past, as well as order placement, coordination of quantities and conditions with suppliers, implementation of goods movement between warehouses and distribution of goods flows in warehouses. This ensures timely and sufficient saturation of goods in stocks to avoid situations where stocks are too large or vice versa: insufficient.
The authors' analysis [7] has concluded that inventory management can reduce inventory costs and avoid inventory surpluses. The use of a multi-agent system for inventory management can offer autonomous or semi-autonomous inventory management, and such systems are used to replenish inventories from online suppliers. The following tools are used in the proposed systems: forecasting algorithms, historical data analysis and stock replenishment algorithms with the possibility to change the methods used.
It is assumed that the manufacturer does not have information on the future demand for the goods, but has information on the previous demand. In order to reduce production stocks in the multi-agent system, it is proposed to use inventory management techniques.
The Inventory management agent has information about the quantity of raw materials and finished goods from the databases. Its main function is to make decisions about when and how much raw materials to buy and when and how much to produce. The agent has knowledge of the possible minimum stock level, i.e., the amount of safety stocks for production capacity; moreover, the agent has a future demand forecasting algorithm. Inventory management agent uses the following methods: ABC analysis algorithm to determine management control, future demand forecasting algorithms for appropriate inventory control, replenishment policy to avoid inventory shortage situations while reducing inventory levels [1], [7].
The essence of the proposed Inventory management agent behaviour algorithm is as follows: it dynamically manages the sufficiency of raw materials and finished products for production and sales. For each raw material and finished product, it calculates: • belonging to the ABC analysis class; • demand forecast; • safety stocks; • ordering points.
After each new request, the demand is compared with the forecast demand. According to the result, the agent recalculates the safety stock and the order point. Belonging to ABC analysis classes and demand forecast are recalculated, including new demand data.
ABC analysis is a basic supply chain management technique often performed by inventory / raw material managers and is the starting point for inventory control. ABC analysis is a categorization system; as a result of this analysis, inventories are classified into three classes, each with a different management control. The ABC analysis is based on the Pareto analysis, which states that 20 % of production represents 80 % of sales.
Demand forecasting in this context is forecasting or estimating the expected demand for products over a certain future period. There is a wide range of forecasting algorithms, but the following forecasting methods were considered and included in the developed multi-agent system: mean method, naive method, weighted moving average and exponential smoothing (1).
The Economic Order Quantity -Reorder Order Policy was chosen for the developed multi-agent system (2).
where C1 -delivery costs per order; C2 -stock storage costs per unit; D -demand.

D. Procurement Agent Development
Raw material management is a process that is considered part of the overall supply chain management process and is focused on solving problems by providing high quality materials at the lowest possible prices to ensure that these materials provide the best benefits in the production process. Raw material management begins with the acquisition and evaluation of raw materials, continues with the use of these materials in the production process, and also includes the assessment of the amount of surplus when these materials are produced in the final product.
The task of raw material management begins with the evaluation and selection of materials required for the production of the final product. The raw material purchasing manager devotes time and effort to finding the right materials, using the company's resources to ensure that they have the right level of quality, and then acting to purchase the materials in the desired quantity.
During the procurement process, the manager also tries to reach an agreement on the best possible price of raw materials that will be used in the production process. In raw material management, the focus is on finding the right materials and using contractual arrangements at a reduced price or volume purchase price to make sure the company has the right level of inventory of materials needed to maintain operations.
The process of raw material management is continuous: constant efforts are made to ensure the appropriate level of quality of raw materials and at the most competitive prices, which allows the manager to find new suppliers and possibly save the company more money in the future. You should always be aware of the opportunities that a company can offer to provide the materials needed for production needs. In this way, the company can also reduce any production disruption that could occur if the ordinary seller is suddenly unable to meet the manufacturer's demand for some reason [8].
The task of the Procurement agent is to order raw materials from the Supplier's agent, as well as to cooperate in matters of prices and delivery options. The goal of the Purchasing agent is to purchase the appropriate raw material at the best possible price (min.) with the best quality (high) with the best delivery time (min.).
Initially, the Procurement agent is idle. When the quantity of raw materials reaches the order point (this point was discussed earlier), the activity of the Procurement agent is activated.
The Procurement agent acts in accordance with its rules of conduct, which shall govern the activities of the agent: • In case of insufficient quantity of raw materials: it sends a request to the Supplier's agent (s) who supplies the raw materials. • If the price and delivery time offered by the Supplier's agent are acceptable, it approves the offer, otherwise rejects it. • When the ordered raw material is sent to the manufacturer, the Purchasing agent enters the standby mode until the next situation with insufficient quantity of raw materials.
It is proposed to continuously collect information from different suppliers: to find new suppliers with better prices and delivery options, or to find an alternative to existing suppliers, so that in the event of unforeseen circumstances where the normal supplier cannot supply raw materials that could delay or interrupt production, there would be an alternative supplier.
The Procurement agent creates a request to purchase raw materials, negotiates with the Supplier's agent on the price and delivery time. The interactions of the above agents take place using a common ManufacturerSupplier ontology [1].
The flow of information between two nodes is low-level. The flow of information is the following two-way process: • Request for raw materials from the Procurement agent to the Supplier's agent. • Supplier's agent offers the requested raw materials to the Procurement agent. • The Procurement agent rejects or accepts the offer from the Supplier agent. • Acknowledgment of receipt and confirmation of shipment of the requested raw materials.
The behaviour of similar agents of other authors has been analysed, and the proposed actions of agents are as follows: • Check the sufficiency of raw materials for production. To be responsible for the choice of suppliers -with minimum raw material prices and the fastest possible deliveries. Generate purchase orders and monitor the delivery of raw materials. • Interact with the supplier on raw materials. Choose the supplier that offers the lowest price. • Purchase raw materials from suppliers. By looking at the demand for raw materials, existing stocks, generate a purchase order.
The following description of the behavioural algorithm is offered to the Procurement agent in the developed multi-agent system: • Purchase raw materials from suppliers. To be responsible for the selection of suppliers: o with minimum raw material prices, with the highest quality, with the fastest deliveries -if there is sufficient time for the delivery of raw materials; o with the minimum delivery time, with the highest quality -if the raw materials are needed for production even at a higher price. • Generate purchase orders and monitor the delivery of raw materials.
Other criteria for the purchase of raw materials may be added in future studies, such as quality, which is not included in this study because the quality of the product is considered to be at the highest level.

V. APPLICATION OF THE DEVELOPED MULTI-AGENT SYSTEM
A multi-agent system based on the description of agent algorithms and communication means has been developed in JADE. Implementation of each agent is provided below.
The production planning algorithm, mentioned in Assembling agent algorithm description, can be illustrated by the following example, where two different product types use the same equipment, but the technological sequence of their production is different.
If only one type of product is produced, the total production time for the first product will be equal to production time of the item, while for the others the total production time will be increased by the hours during which the product waits for the production of previous products.
If the production includes different types of products, then a different production sequence means a different total production time.
Initially, there was a production planning for one product type (product A), then a second product type (product B) was added, and the production planning was rescheduled to the production sequence of the two product types.
The Production agent production planning and re-planning algorithm works as follows: • For all products n all possible different production sequence variants are n! Thus, there is product A and product B, two different options are: 1) product A, then product B or vice versa, 2) product B, then product A. • Check these n! total production times and select the minimal of them.
It should also be noted that in the case of the production of three or more products, the most advantageous sequence will not necessarily be the above sequence and after that the production of the third product, it should have to be recalculated.
Inventory management agent at first provided the ABC analysis. 30 types of products are considered, their price and quantity sold during the year are known. The amount received is the money received for the sale of each product type and the total amount for all goods. Rank is the importance of each product in total product revenue; a percentage of total product revenue and the cumulative importance of the product as a percentage of total revenue were calculated.
The result of the ABC analysis for this example is as follows (30 types of products, 20 % = 6 items): • Class A -58 % of the total revenue is received from the sale of 6 goods. • Class B -32 % of the total revenue is received from the sale of 9 goods, (58 % -90 %); • Class C -10 % of the total revenue is received from the sale of 15 goods, (90 % -100 %).
After the ABC analysis, the Inventory management agent calculates future product demand, safety stocks and order points.
Requests from customers can come at any time and the Seller agent must be able to respond to them in a timely manner, because the level of service depends on the number of completed orders. During the experiment, the Seller agent was able to respond quickly to the requests of Customer1 and Customer2.
The experiment of the Procurement agent dealing with the choice of a supplier: if the required raw materials are available from several of them, then the best option should be chosenthe cheapest raw materials and their delivery, with the same quality. The aim of the experiment was to choose a supplier depending on the priority of the procurement algorithm -the price of the product or the delivery time.
Two suppliers are considered, with the same quality of raw materials, but with different purchase and delivery prices. The Procurement agent has chosen a supplier who offers a better price at the same quality of raw materials.
The following experiment shows the following when different prices are offered: how the agent reacts to different offers. In both cases, the best option for the manufacturing company is chosen, as different offers are expected, they are compared and the best of the alternatives is chosen.

VI. PERFORMANCE MEASUREMENT AND ITS INDICATORS
Performance measurement is defined as the process of determining the efficiency and effectiveness of an operation. Performance is the degree to which customer requirements are met, while efficiency measures how economically a company's resources are used to achieve pre-determined customer satisfaction [9].
Performance measurement should cover all supply chain processes, their results should be quantitative and the indicators should be consistent with the objectives of the supply chain. Appropriate measurements provide an insight into how far a company is from achieving its goal [9].
Supply chain performance indicators [10]- [12] can be divided into two broad categories: qualitative (customer satisfaction, integration of information and material flow, and effective risk management) and quantitative indicators (cost reduction, profit maximization, maximizing request fulfilment, minimizing response time to customer requests, supplier reliability and minimizing lead time).
According to the requirements of the multi-agent system, performance indicators are total production time, inventory storage costs, customer request processing time and service level.
Planning and re-planning production lines, especially if there are capacity constraints on production facilities and long (average monthly) production times, but requests have to be fulfilled within a month and a half, then reducing the total production time is an important task. Its application offered 3 to 14 days revenue in a month.
Inventory management agent application provided inventory decrease up to 29 %, therefore storage costs were significantly reduced.
Inter-node collaboration can be assessed by comparing the time spent by a person and an agent on collaboration. Factors influencing the processing time of a request for a worker are the following: workload, work interruption and health status, comparison of ordered production and stock levels and the human factor. As a result, request processing time can fluctuate: • per person from 10 minutes to 1 working day; • agent for up to 1 minute. The desired level of service of the company is 95 %, which is considered efficient. During the research, all requests were fulfilled, the desired service level was reached.