Mysql error while starting the mysql database in the file tc log – sysadmin

I have found a mysql error while trying to start a mysql database server. The error is relative to the tc.log file at the startup process of mysql database server.

[Note] Recovering after a crash using tc.log
[ERROR] Can't init tc log
[ERROR] Aborting

Why has this mysql error ocurred?

In a quick review of the server, for this case these have been the problems.

• Low hard disc space.
• The log file tc.log was corrupted due to low hard disc space.
• Lack of prevention by not having a resource monitoring for the server.

As sysadmin the first thing we always do is to analyze the log files of the server. At 90% of the cases we can found in this files how to solve the issue. in this case for the mysql error we have found that the problem is at the startup process of mysql database server, specifically in the log system. Possibly because of a lock in the file tc.log

The solution to the mysql error after checking the server

• Free up disk space (this was the trigger of the problem).
• Delete the log file tc.log sudo mv /var/lib/mysql/tc.log /var/lib/mysql/tc_bk.log this way mysql will be able to generate the tc.log file at the next start up.
• As a prevention for future cases, we must to program alarms to notify the client when the server is in a critical situation regarding its resources.

In this case after freeing up the disk space. The server should have been booted up without any problem but one more time instead of booting the server correctly the mysql error persist. So what we proceed to do is a simple file deletion. For security reasons back it up instead of doing a direct deletion.

Basic server failure prevention policies

It is important to always have a minimum of active prevention policies to avoid failures that could leave the server inoperable. Something as basic as setting up threshold for the server resources as memory or disk space could help us avoid a bigger problem like the database getting up corrupted.

In this case because of no prevention policy to avoid server failures we have setup some basics system notifications for the server resources. Notifications for:

• Mysql Database
• Server Memory use threshold
• Hard disk use space threshold
• CPU use threshold

These threshold notification can be easily configured in case of operating under Amazon services from the same Amazon management console. the same applies for Rackspace, Microsoft Azure and similar cloud providers.

Aquí puedes dar clic para visualizar está entrada en español.