I had the same issue and fixed it by making sure the real IP header was set correctly in Nginx and then added RemoteIPHeader X-Forwarded-For in Apache...