Some time ago we faced with the following problem: on-premise Sharepoint 2013 site has 2 authentication zones: Default and Custom. Default authentication zone uses NTLM authentication while Custom uses FBA. Both zones have own host headers (e.g. windows.example.com for Default zone and fba.example.com for Custom). Both host headers were specified in Alternate access mappings of appropriate web application in Sharepoint central administration.
For accessing Sharepoint site remotely nginx was used as reverse proxy between client and internal Sharepoint farm. With this configuration FBA url worked both from within Sharepoint farm (from RDP session) and remotely while Windows url worked only from Sharepoint server and only if we bypass nginx by specifying windows.example.com in hosts file and pointing it to 127.0.0.1 (self IP address). All attempts to login through nginx failed with 401 Unauthorized (and I mean login using custom host header. Logins from RDP session via serve’s name worked).
Investigation showed that nginx doesn’t works well with NTLM authentication (see e.g. How to enable windows authentication through a reverse proxy), so at the end we got rid from nginx in between and configured access to Sharepoint via IP table. If you have solution which works with nginx please share it. Anyway I hope that this information will be helpful.