Apache Redhat 6 make_sock Apache defect
This is a fucker of a defect that affected RedHat 6.3, 6.4, 6.5 - which I've verified. Probably 6.0, 6.1, 6.2 and 6.3 as well. I found this: https://issues.apache.org/bugzilla/show_bug.cgi?id=45863 but it was of no help and was more of a red-herring than anything else.
# httpd -version Server version: Apache/2.2.23 (Unix) Server built: Feb 13 2013 02:41:10
Here is the output that you are confronted with
[Thu Sep 19 14:11:13 2013] [warn] VirtualHost localhost:82 overlaps with VirtualHost localhost:82, the first has precedence, perhaps you need a NameVirtualHost directive [Thu Sep 19 14:11:13 2013] [warn] VirtualHost localhost:81 overlaps with VirtualHost localhost:81, the first has precedence, perhaps you need a NameVirtualHost directive (98)Address already in use: make_sock: could not bind to address 127.0.0.1:81 no listening sockets available, shutting down Unable to open logs
Doing an lsof, netstat, telnet or anything else you can think of on port 81 tells you nothing is listening. OK so why the error message !
The strace is telling its trying to bind TWICE to the same port! WTF.
32709 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0 32709 setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0 32709 setsockopt(3, SOL_TCP, TCP_NODELAY, [1], 4) = 0 32709 bind(3, {sa_family=AF_INET, sin_port=htons(81), sin_addr=inet_addr("127.0.0.1")}, 16) = 0 32709 listen(3, 511) = 0 32709 setsockopt(4, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0 32709 setsockopt(4, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0 32709 setsockopt(4, SOL_TCP, TCP_NODELAY, [1], 4) = 0 32709 bind(4, {sa_family=AF_INET, sin_port=htons(81), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EADDRINUSE (Address already in use)
The solution. The trouble makers were found in /etc/hosts as follows
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
Once we had removed 'localhost' and 'localhost.localdomain' from the ::1 definition line, apache/httpd could start successfully.
UPDATE: In light of https://issues.apache.org/bugzilla/show_bug.cgi?id=52884
In my configuration I had this
Listen localhost:81
If I had change it to this it would have also made the problem go away.
Listen 127.0.0.1:81