Analysis Of Server Logs To Determine The Right Timing For Server Maintenance Using A Single Node And Multi-Node Hadoop Environment

The log files that are continuously recorded are one of the most important information generated by web servers.Hadoop is an open source application platform that works on large amounts of data using hardware clusters. Web server logs are large-volume plain text files that are meaningfully structured. The servers use customized operating systems for resource sharing. It is extremely important that, some precautions like security updates are formerly planned by monitoring the load status of the servers over time. Therefore, it is necessary to determine when the server workload is minimal. In the Hadoop environment, the Hadoop file system is used to store the log files and the results are appreciated using the map and reduce functions. In this study, the Hadoop MapReduce programming model was used with two mode for analyzing weblogs. These modes are single node environment mode and distributed environment mode. By using our approach, we have determined how many connections made to the server during the day and we have tried to find the best time for maintenance.