There could be a few reasons why the Redis Sentinel failover isn't working. Some common causes include:
Network Issues: Failover may be failing due to network disruptions between your Sentinels and Redis servers.
Sentinel Configuration: If the configuration of your sentinels is incorrect, it could lead to failover issues. This can occur if the sentinel monitor or sentinel down-after-milliseconds configurations are not set appropriately.
Insufficient Sentinel Instances: For a failover to happen correctly, it's recommended to have at least three sentinel instances. Having less than three might result in failover issues.
Quorum Not Met: The failover process also requires a majority of sentinel nodes to agree on the master being down (quorum). If this condition is not met, the failover will not take place.
Solution - Here's How To Resolve It
Based on the potential causes outlined above, here are some possible solutions:
Check Network Connectivity: Ensure that all connections between your sentinels and Redis servers are stable and reliable.
Review Sentinel Configuration: Check your sentinel configuration for any mistakes. Make sure that the sentinel monitor command has been correctly set with the right master name, IP address, port, and quorum. Also, verify that the sentinel down-after-milliseconds setting is properly configured based on your requirements.
Increase Sentinel Instances: If you have fewer than three sentinel instances, consider adding more. A minimum of three sentinel instances ensures a failover can still happen even if one of the sentinels fails.
Ensure Quorum Is Met: Check that the number of sentinels required for a quorum has been correctly configured and is being met. This number should be less than the total number of sentinel instances to allow failover even when some sentinels are unreachable.