Bug 58593 - mod_proxy wrong decision between http and ws
Summary: mod_proxy wrong decision between http and ws
Status: RESOLVED FIXED
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_proxy (show other bugs)
Version: 2.4.17
Hardware: All All
: P2 normal (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-11-06 11:43 UTC by Tabby
Modified: 2023-01-30 10:54 UTC (History)
2 users (show)



Attachments
mod_proxy_wstunnel fall through http (2.78 KB, patch)
2015-11-30 17:41 UTC, Yann Ylavic
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tabby 2015-11-06 11:43:14 UTC

    
Comment 1 Yann Ylavic 2015-11-06 11:54:54 UTC
Wrong description often leads to wrong decision ;)
Comment 2 Tabby 2015-11-06 12:03:51 UTC
Hi Yann,
not sure if this is a bug, anyway I don't know how to handle this case.

Scenario:
We have to serve both http and ws requests under the same URI, through an apache reverse proxy.

I thought this would work:
<Location /ws/live/>
    ### Settings for WebSocket
    ProxyPass        ws://vip-be/wwgw/var/ disablereuse=on  timeout=30

    ### Settings for HTTP
    ProxyPass        http://vip-be/wwgw/var/ smax=0  ttl=15
    ProxyPassReverse http://vip-be/wwgw/var/
</Location> 

but in case of websocket requests it seems mod_proxy always selects http handler, instead of ws, despite the right choice of scheme done by the core.

Here are the logs:

[core:debug]  protocol.c(1840): [client 172.26.130.177:59719] select protocol from , choices=websocket for server wstest
[proxy:debug]  mod_proxy.c(1160): [client 172.26.130.177:59719] AH01143: Running scheme http handler (attempt 0)
[proxy:debug]  proxy_util.c(2160): AH00942: HTTP: has acquired connection for (vip-be)
[proxy:debug]  proxy_util.c(2213): [client 172.26.130.177:59719] AH00944: connecting http://vip-be/wwgw/var/5 to vip-be:80
[proxy:debug]  proxy_util.c(2422): [client 172.26.130.177:59719] AH00947: connected /wwgw/var/5 to vip-be:80
[proxy:debug]  proxy_util.c(2799): AH02824: HTTP: connection established with 172.26.110.165:80 (vip-be)
[proxy:debug]  proxy_util.c(2965): AH00962: HTTP: connection complete to 172.26.110.165:80 (vip-be)
[proxy:debug]  proxy_util.c(2175): AH00943: http: has released connection for (vip-be)

Am I missing something or doing it wrong?
Thanks

ST
Comment 3 Tabby 2015-11-30 09:48:30 UTC
Sorry for bothering,
any advice on this?

Thanks
ST
Comment 4 Ruediger Pluem 2015-11-30 14:40:33 UTC
(In reply to Tabby from comment #2)
> Hi Yann,
> not sure if this is a bug, anyway I don't know how to handle this case.
> 
> Scenario:
> We have to serve both http and ws requests under the same URI, through an
> apache reverse proxy.
> 
> I thought this would work:
> <Location /ws/live/>
>     ### Settings for WebSocket
>     ProxyPass        ws://vip-be/wwgw/var/ disablereuse=on  timeout=30
> 
>     ### Settings for HTTP
>     ProxyPass        http://vip-be/wwgw/var/ smax=0  ttl=15
>     ProxyPassReverse http://vip-be/wwgw/var/
> </Location> 
> 

The configuration is wrong. You can only have one ProxyPass in a Location block. If you really need to serve normal HTTP requests and Websockets from the same URL you need to do some mod_rewrite magic looking for the Upgrade header.
Comment 5 Tabby 2015-11-30 15:15:47 UTC
Hi,
that's exactly how I did as a workaround.
---------------------------------------------------------------------------
  RewriteEngine On

  # default protocol = ws
  RewriteRule ^(.*)$ - [env=proto:ws]

  # if request is plain http, set protocol = http
  RewriteCond %{HTTP:Upgrade} !websocket [NC]
  RewriteRule ^ - [env=proto:http]

  RewriteCond %{REQUEST_URI} ^/ws/live/ [NC]
  RewriteRule ^/ws/live/(.*)  %{ENV:proto}://vip-be/wwgw/var/$1 [P,QSA,L]
---------------------------------------------------------------------------

Anyway, this way I'm forced to use mod_rewrite "Proxy" flag and this leads to performance penalties, 'cause it won't use backend connection pooling (default worker being used).

Is should be really nice if I could use Env interpolation even in the scheme part of the URL (something like: ProxyPass  %{ENV:proto}://vip-be/wwgw/var/ ), but the doc states that isn't supported.

So, is there a better solution?

Thanks.
ST
Comment 6 Yann Ylavic 2015-11-30 17:27:42 UTC
You could probably use something like:
    <Proxy http://vip-be/wwgw/var/>
        ....
    </Proxy>
in your configuration, so that this proxy http worker can be reused.
Comment 7 Yann Ylavic 2015-11-30 17:31:46 UTC
(In reply to Yann Ylavic from comment #6)
> so that this proxy http worker can be reused.

I meant connections for this worker can be reused (kept alive)...
Comment 8 Yann Ylavic 2015-11-30 17:41:35 UTC
Created attachment 33314 [details]
mod_proxy_wstunnel fall through http

Possibly a patch like this one could allow ProxyPass to either ws(s) or http(s) on the same URL, depending on the Upgrade header, but it is probably a hack :/

We'd rather use the new Protocols negotiation to enable/decline mod_proxy_wstunnel for a request, if that's possible/compatible.
Comment 9 Eric Covener 2015-11-30 18:01:30 UTC
(In reply to Yann Ylavic from comment #8)
> Created attachment 33314 [details]
> mod_proxy_wstunnel fall through http
> 
> Possibly a patch like this one could allow ProxyPass to either ws(s) or
> http(s) on the same URL, depending on the Upgrade header, but it is probably
> a hack :/
> 
> We'd rather use the new Protocols negotiation to enable/decline
> mod_proxy_wstunnel for a request, if that's possible/compatible.

I vaguelly recall Jim trying something here for this requirement, but it was not a 100% match -- but the thread died.
Comment 10 Tabby 2015-12-01 09:33:24 UTC
Sorry, but I'm a bit confused.
Since using the same URL for ws(s) and http(s) seems to be perfectly legal (although I'd always go with distinct URL patterns), I'm not sure about what you are suggesting to do.

At the moment, is there a valid alternative to this?
---
RewriteRule ^/ws/live/(.*)  %{ENV:proto}://vip-be/wwgw/var/$1 [P,QSA,L]
<Proxy http://vip-be/wwgw/var>
  ProxySet smax=0 ttl=15
</Proxy>
---

Furthermore: is it necessary, in your opinion, to add a <Proxy> block also for ws, in order to keep its connection alive or is it worthless because of its long-living nature?

Thanks
ST
Comment 11 Yann Ylavic 2015-12-01 12:46:48 UTC
(In reply to Tabby from comment #10)
> 
> At the moment, is there a valid alternative to this?
> ---
> RewriteRule ^/ws/live/(.*)  %{ENV:proto}://vip-be/wwgw/var/$1 [P,QSA,L]
> <Proxy http://vip-be/wwgw/var>
>   ProxySet smax=0 ttl=15
> </Proxy>
> ---

No currently there isn't an alternative, mod_proxy can't work either ws: or http: on the same path.

The patch I proposed above may help though (not tested), maybe you could try it (instead of the RewriteRules, it should work without).

> 
> Furthermore: is it necessary, in your opinion, to add a <Proxy> block also
> for ws, in order to keep its connection alive or is it worthless because of
> its long-living nature?

Once the connection is upgraded to websocket, it can't (and must not) be reused/kept-alive.
mod_proxy_wstunnel takes care of that already, whatever is set for disablereuse (if the proxy worker is declared).
Comment 12 Yann Ylavic 2015-12-01 13:36:36 UTC
(In reply to Yann Ylavic from comment #11)
> 
> The patch I proposed above may help though (not tested), maybe you could try
> it (instead of the RewriteRules, it should work without).

Please note that you may need to place "ProxyPass ws://..." before "ProxyPass http://..." for it to work.
Comment 13 Mina Galić 2023-01-26 09:21:23 UTC
I know this is an old issue, with a well known workaround.
But I'm still wondering if anyone is looking into fixing this on a mod_proxy level.

Having applications with HTTP and WebSockets is pretty much the standard these days, and none of them can benefit from mod_proxy's full featureset, thanks to this design constraint, and this workaround.
Comment 15 Yann Ylavic 2023-01-27 11:48:12 UTC
Since 2.4.47 I think there is no need for two ProxyPass directives, a single:
  ProxyPass /some/path/ https://backend/some/path/ upgrade=websocket ...
should proxy for both ws and http (up to the ws handshake) on that path.

Did someone tried this?

Using "ProxyWebsocketFallbackToProxyHttp on" would restore the old behaviour thus two ProxyPass needed (supposedly).
Comment 16 Yann Ylavic 2023-01-27 11:55:36 UTC
(In reply to Yann Ylavic from comment #15)
> 
> Using "ProxyWebsocketFallbackToProxyHttp on" would restore the old behaviour
> thus two ProxyPass needed (supposedly).

ProxyWebsocketFallbackToProxyHttp *off* would restore the old behaviour, the default is "on" (websocket upgrade is fully handled by mod_proxy_http only).
Comment 17 Mina Galić 2023-01-27 12:03:00 UTC
if this has been possible since 2.4.47, why is the documentation still taking about mod_rewrite? https://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html
Comment 18 Yann Ylavic 2023-01-27 12:42:45 UTC
This documentation is about mod_proxy_wstunnel, and using mod_proxy_wstunnel requires the mod_rewrite workaround still.
With 2.4.47 and newer it's possible to proxy the websocket protocol (ws and wss) by using/loading mod_proxy_http only and adding upgrade=websocket to the ProxyPass line.
Admittedly this is not well documented, that's why I'm trying to make this more clear here. I'll try to update the general mod_proxy documentation about this (https://httpd.apache.org/docs/2.4/mod/mod_proxy.html) since it talks about the ProxyPrass upgrade= parameter as something related to mod_proxy_wstunnel only.
Help on documentation and other always welcome though..
Comment 19 Yann Ylavic 2023-01-27 16:56:37 UTC
New note added to https://httpd.apache.org/docs/trunk/mod/mod_proxy.html#protoupgrade (hth..).
Comment 20 Mina Galić 2023-01-28 11:30:30 UTC
it works.
thanks!
Comment 21 Mina Galić 2023-01-28 15:46:09 UTC
one more question: could it be that the mod_rewrite workaround will work on 2.4.47+ without mod_proxy_wstunnel loaded, too?
Comment 22 Yann Ylavic 2023-01-29 11:10:34 UTC
I think it does work because the workaround sets the ws(s) scheme for mod_proxy, which mod_proxy_http 2.4.47+ treats as/like "upgrade=websocket" (for compatibility precisely).
Comment 23 Yann Ylavic 2023-01-29 11:44:10 UTC
I mean with mod_proxy_http 2.4.47+ (and without mod_proxy_wstunnel), both:
  ProxyPass "/" "http://example.com/some/ws/capable/path/" upgrade=websocket
and:
  ProxyPass "/" "ws://example.com//"
work the same.

And mod_proxy_http will internally do the same as:
  RewriteCond %{HTTP:Upgrade} websocket [NC]
  RewriteCond %{HTTP:Connection} upgrade [NC]
without the need of mod_rewrite. If no websocket is negotiated by the client, the request will be forwarded by using/enforcing HTTP as usual.
Comment 24 Yann Ylavic 2023-01-29 11:46:07 UTC
Messed up examples:
>   ProxyPass "/" "http://example.com/some/ws/capable/path/" upgrade=websocket
> and:
>   ProxyPass "/" "ws://example.com//"
should read:
  ProxyPass "/" "http://example.com/" upgrade=websocket
and:
  ProxyPass "/" "ws://example.com/"
Comment 25 Mina Galić 2023-01-29 23:14:07 UTC
i think we can close this bug then.?
Comment 26 Yann Ylavic 2023-01-30 10:54:56 UTC
Fixed in 2.4.47.