IMPLEMENTATION OF TECHNOLOGY FOR IMPROVING THE QUALITY OF SEGMENTATION OF MEDICAL IMAGES BY SOFTWARE ADJUSTMENT OF CONVOLUTIONAL NEURAL NETWORK HYPERPARAMETERS

Background. The scientists have built effective convolutional neural networks in their research, but the issue of optimal setting of the hyperparameters of these neural networks remains insufficiently researched. Hyperparameters affect model selection. They have the greatest impact on the number and size of hidden layers. Effective selection of hyperparameters improves the speed and quality of the learning algorithm. It is also necessary to pay attention to the fact that the hyperparameters of the convolutional neural network are interconnected. That is why it is very difficult to manually select the effective values of hyperparameters, which will ensure the maximum efficiency of the convolutional neural network. It is necessary to automate the process of selecting hyperparameters, to implement a software mechanism for setting hyperparameters of a convolutional neural network. The author has successfully implemented the specified task. Objective . The purpose of the paper is to develop a technology for selecting hyperparameters of a convolutional neural network to improve the quality of segmentation of medical images.. Methods. Selection of a convolutional neural network model that will enable effective segmentation of medical images, modification of the Keras Tuner library by developing an additional function, use of convolutional neural network optimization methods and hyperparameters, compilation of the constructed model and its settings, selection of the model with the best hyperparameters. Results. A comparative analysis of U-Net and FCN-32 convolutional neural networks was carried out. U-Net was selected as the tuning network due to its higher quality and accuracy of image segmentation. Modified the Keras Tuner library by developing an additional function for tuning hyperparameters. To optimize hyperparameters, the use of the Hyperband method is justified. The optimal number of epochs was selected - 20. In the process of setting hyperparameters, the best model with an accuracy index of 0.9665 was selected. The hyperparameter start_neurons is set to 80, the hyperparameter net_depth is 5, the activation function is Mish, the hyperparameter dropout is set to False, and the hyperparameter bn_after_act is set to True. Conclusions. The convolutional neural network U-Net, which is configured with the specified parameters, has a significant potential in solving the problems of segmentation of medical images. The prospect of further research is the use of a modified network for the diagnosis of symptoms of the coronavirus disease COVID-19, pneumonia


Introduction
One of the important components of the medical image processing process is segmentation. This process has become widely used during the study of medical facilities. Segmentation provides the ability to divide medical images into parts that correspond to certain features. These signs are used in the processes of detection, diagnosis and treatment of diseases of various degrees of severity. Segmentation reduces the error of medical tests and helps establish an accurate diagnosis. Segmentation improves the quality of medical image analysis. In recent years, the goal of research is to fully automate the process of medical image segmentation. Thanks to the achievement of the specified goal, the reliability of determining the signs of various diseases of the human body should increase. Another goal of segmentation is to reduce the time it takes to diagnose and treat diseases due to their rapid spread, as well as the emergence of a large number of new infections in recent years. This circumstance becomes especially relevant during the height of the COVID-19 coronavirus disease.
This infectious disease has become one of the most dangerous problems of humanity in the 21st century. The first person in Ukraine died of the infection on March 13, 2020 in Chernivtsi. According to official data, as of April 30, 2023, 5,518,614 people were infected in Ukraine, which is 13.4% of the population of Ukraine. 112,023 deaths were recorded, which is 2% of the population. The most affected regions were the city of Kyiv and Kyiv region, Odesa, Dnipropetrovsk, Lviv and Kharkiv regions. Currently, the main method of laboratory diagnosis of COVID-19 in Ukraine is PCR -a test that allows you to determine the presence/absence of genetic material of the virus in a swab from the mucous membrane of the nasopharynx or mouth. The disadvantages of this method of determining the signs of the coronavirus disease COVID-19 include a sufficiently high level of false negative results. The World Health Organization recommends the use of computed tomography of the chest as an additional or alternative method of diagnosing COVID-19. In the first studies before the advent of machine learning, SVM, Random Forest, and K-means Clustering algorithms were used. But in recent research, scientists use deep learning methods to solve segmentation problems. Deep learning performed better than SVM, Random Forest and K-means Clustering algorithms. Therefore, modern methods use exactly this approach. Scientists have built effective convolutional neural networks in their research, but the issue of optimal setting of the hyperparameters of these neural networks remains insufficiently researched. Hyperparameters affect model selection. They have the greatest impact on the number and size of hidden layers. Effective selection of hyperparameters improves the speed and quality of the learning algorithm. It should also be noted that the hyperparameters of the convolutional neural network are interconnected. That is why it is very difficult to manually select the effective values of hyperparameters, which will ensure the maximum efficiency of the convolutional neural network. The main task of the research is the development of technology that will improve the quality of effective selection of convolutional neural network parameters and automate the selection of hyperparameters based on the example of the selected model.

Development of technology to improve the quality of segmentation of medical images.
To improve the quality of segmentation of medical images, it is necessary to choose the most effective neural network. As a result of the study, the architectures of two convolutional neural networks FCN-32 and U-Net were analysed. The first version of the FCN convolutional neural network architecture was developed by Evan Shelgamer, Jonathan Long, and Trevor Darrell and proposed in the paper "Fully Convolutional Networks for Semantic Segmentation" [1]. The architecture of the specified convolutional network is presented in Fig.1. As a result of the study, it was established that the indicated convolutional neural network has a significant drawback. This disadvantage consists in the loss of information on the final layer due to a 32-fold reduction in sampling. In the future, it is very difficult to perform a 32x upsampling using small information. The next step of the research was the analysis of the accuracy and efficiency of the U-Net convolutional neural network. The architecture of this network is presented in Fig.2. The architecture of the basic version of the convolutional neural network U-Net was developed from the very beginning of its existence to implement the segmentation of medical images at the Faculty of Computer Science of Freiburg [2]. The advantages of this network include the ability to work with fewer images for training. This feature of the U-Net convolutional neural network is a significant advantage over analogues that process large data sets and require significant time resources in the learning process. At the same time, the U-Net convolutional neural network ensures high accuracy and quality of segmentation. The specified efficiency indicators are provided due to the increase in dimensionality and the existence of a large number of feature channels. The specified features of the U-Net convolutional neural network allow spreading information to layers of higher resolution. As a result of these features, parts of the network with expansion and contraction acquire a symmetrical structure. It should also be noted that the U-Net convolutional neural network looks like the letter U. Its compactness is one of the advantages of this network. It uses only convolution layers and does not use fully connected layers. In the research of scientists, many modifications of U-Net were presented, which ensured high-quality segmentation of images. In particular, several versions of the specified convolutional neural network were presented, which helped to determine the symptoms of the coronavirus disease COVID-19 [3][4][5][6][7][8]. However, the issue of effective selection of convolutional neural network hyperparameters remains insufficiently elucidated.
In the study, the Keras Tuner library will be used to select hyperparameters. This framework is developed by Google specifically for the Keras library as part of Tensorflow 2.0. The advantages of Keras Tuner include free, flexible and easy to use, as well as the possibility of modification, further development of new versions in order to improve the efficiency of setting the hyperparameters of the convolutional neural network. Therefore, to use this library, you need to use the command import keras_tuner from tensorflow import keras. In the course of the study, the Keras Tuner library module was modified by developing an additional function that improves the quality of setting hyperparameters of a convolutional neural network. By default, the U-Net architecture has 8 hyperparameters with the following values: input_size is (512, 512, 1), start_neurons is 64, net_depth is 4, output_classes is 1, dropout is False, bn_after_act is False, activation is Mish, pretrained_weights is None. The input_size hyperparameter allows you to define the size of the input image. The output_classes hyperparameter stores the number of output classes. The pretrained_weights hyperparameter specifies the path to the weights of the pretrained model. Taking into account the fact that in the studies the specified parameters acquire different values depending on the problem, we will focus on adjusting other hyperparameters. The hyperparameter start_neurons is responsible for the number of neurons. By default in the U-Net network, it is 64. The net_depth hyperparameter defines the depth of the network. By default in the U-Net network, it is equal to 4. The dropout hyperparameter, which acquires the value True or False, is responsible for using the regularization method of artificial neural networks in order to reduce the overtraining of the network. By default in the U-Net network it is False. The activation hyperparameter defines the activation function. The Mish activation function is used by default. The advantages of this function include its smoothness, which allows optimizing the gradient flow. It should also be noted that Mish is a self-regulating activation function. In the created function, it is possible to choose one of the activation functions -Mish, ELU or LReLU. Currently, ReLU as an activation function is most often used in the construction of convolutional neural networks. Its main advantage is low complexity. During a direct pass, negative values become 0, and positive values remain unchanged. In the reverse pass, the value of the derivative is 0 for negative values, and for positive values it becomes 1. However, it also has significant drawbacks. In the ReLU function, neurons that are not initially activated can never be activated. Those neurons whose inputs received negative values will also be added to them. The gradient descent algorithm will not be able to adjust the weights of such neurons. Another activation function used from the function list is called ELU. The peculiarity of this function is that it uses a logarithmic curve for negative values. This feature makes it possible to achieve the saturation of neurons in this interval, to reduce the variability of the data distributed through the convolutional neural network. Another activation function used from the function list is called LReLU. In contrast to ReLU, it leaves small negative values of neurons and preserves for them the possibility of being activated. This is achieved due to the coefficient α, which acquires small values. The value of 0.01 is most often used in research. At the same time, the computational complexity is also insignificant, and in the reverse pass, the use of the angular coefficient makes it possible to achieve nonzero gradients at negative values. This feature allows scales to be updated. Programmatically, the most effective activation function is selected using the operator activation = hp.Choice(name = 'activation', values = ['mish', 'elu', 'lrelu'], ordered = False). The hyperparameter number of neurons is selected in the range from 16 to 128 with a step of 16. This is done programmatically using the operator start_neurons = hp.Int(name = 'start_neurons', min_value = 16, max_value = 128, step = 16). The depth of the network is chosen in the range from 2 to 8. The dropout hyperparameter, which is responsible for regularization in the convolutional neural network in order to reduce its overtraining, is defined by the operator dropout = hp.Boolean(name = 'dropout', default = False). The developed function is compiled using the Adam optimization method. Its advantages include an efficient combination of root mean square propagation and gradient descent with momentum algorithms. In the Adam method, small memory requirements are combined with high computational efficiency. This method was also used because it showed high classification accuracy for the CIFAR-10 data set -98.5%. The use of other optimization methods showed their lower efficiency. For the SGD optimization method, the accuracy rate was 92.85%, for the Momentum optimization method, the accuracy rate was 95.52%, for the RMSProp optimization method, the accuracy rate was 97.2%. Metrics f1, precision, recall, iou were used for compilation. Intersection over Union (iou) is an evaluation metric used to measure the accuracy of an object classifier on a specific data set. Keras Turner has several built-in methods to optimize hyperparameters. In the study, the optimization methods RandomSearch, BayesianOptimization and HyperBand were tested for effectiveness. A study conducted on U-Net hyperparameters showed that the RandomSearch optimization method has the lowest performance. The specified method only chooses the value of the hyperparameter randomly, iterating over all possible options from the specified range. The optimization method BayesianOptimization showed better performance due to the fact that it does not choose combinations of hyperparameters randomly like RandomSearch. This method takes into account already tested combinations and uses the information to select the next combination for further optimization. The max_trials parameter specifies the maximum total number of trials. By default, it gets a value of 10. The alpha parameter determines the expected amount of noise in the images. By default, its value is 0.0001. The study of the Hyperband hyperparameter optimization method showed that this method is significantly more efficient in terms of time resource consumption than the RandomSearch and BayesianOptimization optimization methods. One of the key parameters of this method is max_epochs, which defines the maximum number of epochs for training one model. By default, it is 100. But in the study, its value is 20, considering the convolutional neural network architecture of U-Net. In the process of setting hyperparameters of a convolutional neural network in Keras Tuner, the search method is used instead of the standard fit method. As a result of 50 runs of the built system, different models with different accuracy results were obtained. It was concluded that hyperparameters start_neurons and net_depth have the greatest influence on the accuracy of the network. For the worst models, which received accuracy values of 0.4954 and 0.5179, these indicators were respectively start_neurons=32 and net_depth=3, start_neuron=96 and net_depth=3. For the best models, the indicated accuracy indicators took on values of 0.9361 and 0.9665. For the best model, the start_neurons hyperparameter was 80, and the net_depth was 5. For the second most accurate model, the start_neurons hyperparameter was 32, and the net_depth hyperparameter was 5. During the setup process, Mish or ELU were chosen as the activation function by the system. In the best model, the activation function of Mish was determined. The ELU activation function was determined in the second most accurate model. In the most accurate model, the following values of other hyperparameters were obtained: hyperparameter dropout equals False, hyperparameter bn_after_act equals True. In the second most accurate model, the following values were obtained: the dropout hyperparameter is False, and the bn_after_act hyperparameter is True.

Conclusion
In the course of the study, the use of the U-Net convolutional neural network for solving the problems of medical image segmentation was substantiated. Compared to FCN-32 convolutional neural network, it has better characteristics and significant potential for medical image segmentation. The Keras Turner library has been modified by developing an additional function for efficient tuning of U-Net hyperparameters. The use of the Adam optimization method to improve the performance of the U-Net convolutional neural network is substantiated. The advantages of the specified function in classification accuracy in comparison with SGD, Momentum and RMSProp optimization methods are given. To optimize hyperparameters, the use of the Hyperband method is justified. The optimal number of epochs was selected, which is equal to 20. In the process of setting the hyperparameters, the best model with an accuracy index of 0.9665 was selected, the start_neurons hyperparameter is 80, the net_depth hyperparameter is 5, the activation function is Mish, the dropout hyperparameter is set to False, and the bn_after_act hyperparameter is set to True. The U-Net convolutional neural network, configured with the specified parameters, has significant potential in solving medical image segmentation problems. The prospect of further research is the use of a modified network for the diagnosis of symptoms of the coronavirus disease COVID-19, pneumonia, cancer and other complex medical diseases.