Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
HTTP persistent connection
#1
When I tried

Macro
Code:
Copy      Help
str s
rep 10
,Http h.Connect("www.xxx.yyy" "user" "password")
,h.Post("test/test.php" "name1=value1&name2=value2" s)
,out s

the time used is high.

I tried also to use several functions in parallel.

It can be done faster?

note: in wikipedia (Hypertext Transfer Protocol):

In HTTP/0.9 and 1.0, the connection is closed after a single request/response pair. In HTTP/1.1 a keep-alive-mechanism was introduced, where a connection could be reused for more than one request.

Such persistent connections reduce lag perceptibly, because the client does not need to re-negotiate the TCP connection after the first request has been sent.

Version 1.1 of the protocol made bandwidth optimization improvements to HTTP/1.0. For example, HTTP/1.1 introduced chunked transfer encoding to allow content on persistent connections to be streamed, rather than buffered. HTTP pipelining further reduces lag time, allowing clients to send multiple requests before a previous response has been received to the first one. Another improvement to the protocol was byte serving, which is when a server transmits just the portion of a resource explicitly requested by a client.
#2
I don't know much about HTTP authentication, speed, etc.

QM uses wininet functions and HTTP 1.1. I think they don't close connection after each request but not sure. Try to add INTERNET_FLAG_KEEP_CONNECTION as 4-th argument of Post.

You can read about wininet functions in MSDN library.
#3
With INTERNET_FLAG_KEEP_CONNECTION works better but still slow.

I found this:

http://www.codeproject.com/KB/IP/asyncwininet.aspx
http://www.codeproject.com/KB/cpp/WinInetRunScript.aspx

Can it be used in QM?
#4
It is not related to internet connection speed. It means that the thread can do other things while wininet functions are connecting and transferring data.
#5
I also found an activex dll: http://www.coalesys.com/products/httpcl ... efault.asp

Do you know if uses different functions that QM?

In affirmative case, could you add an example of several Post in parallel to test if is faster?
#6
Slower. Every time about 400 ms. QM - first time 400 ms then 190.

GET example
Macro
Code:
Copy      Help
out
typelib CSHttpClientLib {12CB8C40-6A6B-11D0-A74C-444553540000} 1.0
CSHttpClientLib.CSHttpClient c._create
c.RequestURL="http://www.google.com"
c.Execute("GET")
out c.ResponseBody(0)

POST example
Macro
Code:
Copy      Help
out
typelib CSHttpClientLib {12CB8C40-6A6B-11D0-A74C-444553540000} 1.0
CSHttpClientLib.CSHttpClient c._create
c.RequestURL="http://www.quickmacros.com/form2.php"
str s="a=b&c=d"
VARIANT v=s
c.RequestBody=v
c.RequestHeaders=_s.format("Content-Type: application/x-www-form-urlencoded[]Content-Length: %i[]Cache-Control: no-cache[]" s.len)
rep 5
,int t1=timeGetTime
,c.Execute("POST")
,out timeGetTime-t1
,out c.ResponseBody(0)

Don't know how to set password. Maybe in URL, but did not test.
#7
How can I measure the time of the Function_test?

Function Function_test
Code:
Copy      Help
rep 10
,mac("test_post" "" _s)
#8
Macro
Code:
Copy      Help
ARRAY(int) a
int t1=timeGetTime
rep 10 ;;max 64
,a[]=mac("test_post" "" _s)
;wait until all threads ended
if(WaitForMultipleObjects(a.len &a[0] 1 INFINITE)=WAIT_FAILED) end _s.dllerror
out timeGetTime-t1
#9
Thank you very much.

Is not possible for >64 threads?
#10
WaitForMultipleObjects supports max 64 handles.
You should not use many threads because each thread gets 1 MB of memory for stack. For example, on 256 MB RAM PC you cannot have more than about 200 threads.
#11
Almost without differences...


64 Post threads

Without INTERNET_FLAG_KEEP_CONNECTION

4943 ms
5712 ms
5218 ms
5567 ms

With INTERNET_FLAG_KEEP_CONNECTION

5053 ms
5198 ms
4903 ms
5251 ms
#12
It seems that the solution is here: http://forums.techarena.in/tips-tweaks/990999.htm

with 100 Post (time of each individual post 5-15 seconds):

before edit registry ----> 12 minutes
after edit registry -----> 4 minutes


Questions:

1) Can I do it without edit registry?
2) Can I Know the number of simultaneous connections of server for optimal performance?
3) How can I add a timeout of 1 minute in Post.

Note: sometimes any post time is very high (4-6 minutes) and if make the same post in that moment with IE the time is 5-15 seconds. Do you know why?
#13
can answer only question 3.

Member function Http.PostWithTimeout
Code:
Copy      Help
function# timeout_s $action $data [str&responsepage] [$headers] [inetflags] [str&responseheaders] [&unused]

;Posts web form data. Returns 1 on success, 0 on failure. Error on timeout.
;This function cannot post files. To post files use PostFormData.

;timeout_s - timeout in seconds.
;data - urlencoded string, eg "name=John+Smith&email=j.smith@xxx.com".
;action, responsepage, headers, inetflags, responseheaders - the same as with PostFormData.
;unused - used internally.

;See also: <Http.PostFormData>.


if(!&unused)
,if(!m_hi) end ES_INIT
,__HInternet _hi; int th _ret
,th=mac("__Http_PostWithTimeout" "" &this action data &responsepage headers inetflags &responseheaders &_hi &_ret)
,wait timeout_s H th
,err
,,InternetCloseHandle _hi; _hi.handle=0 ;;ends thread
,,wait 0 H th ;;because the thread uses this. ~0.5ms
,,end _error
,ret _ret

lpstr sh="Content-Type: application/x-www-form-urlencoded"
if(empty(headers)) headers=sh
else if(findrx(headers "(?i)^Content-Type *:" 8)<0) headers=_s.from(sh "[]" headers)

unused=HttpOpenRequest(m_hi "POST" action 0 0 0 INTERNET_FLAG_RELOAD|inetflags 0); if(!unused) ret Error
if(!HttpSendRequest(unused headers -1 data len(data))) ret Error

if(&responseheaders and !GetResponseHeaders(unused responseheaders)) ret Error

if(&responsepage) ret Read(unused responsepage)
ret 1

Function __Http_PostWithTimeout
Code:
Copy      Help
;\
function Http&h $action $data str&responsepage $headers inetflags str&responseheaders &handle &_ret

_ret=h.PostWithTimeout(0 action data responsepage headers inetflags responseheaders handle)
#14
Can Winhttp be better than Wininet (http://msdn.microsoft.com/en-us/magazine/cc716528.aspx) to do this?

Can you add an example of

rep 10
,mac("test_post" "" _s)

using winhttp (this API has Timeout, Asynchronous...) to test?

Thanks in advance.
#15
WinHTTP is available on XP SP1 and later. Is it OK?
#16
http://msdn.microsoft.com/en-us/library/aa384276(VS.85).aspx

download

http://www.dlldll.com/winhttp.dll_download.html
#17
Much to learn. I don't think it will be better than you already have in QM.
#18
http://www.autoitscript.com/forum/index ... opic=84133
#19
Need declarations?
More Windows API declarations for QM
example
Macro
Code:
Copy      Help
WINAPI2.WinHttpSendRequest
#20
No, I wonder if Winhttp functions are quicker than Wininet

Http.PostWithTimeout uses 2 threads and I need run more than 100 Post...
#21
I'm trying understand the function InternetSetOption

How can I create a Http member function to set:

INTERNET_OPTION_MAX_CONNS_PER_SERVER
INTERNET_OPTION_MAX_CONNS_PER_1_0_SERVER
INTERNET_OPTION_CONNECT_TIMEOUT
INTERNET_OPTION_RECEIVE_TIMEOUT
INTERNET_OPTION_FROM_CACHE_TIMEOUT

another question:

if set INTERNET_OPTION_RECEIVE_TIMEOUT is necessary to use Http.PostWithTimeout?
#22
The timeout options don't work, that is why I used another thread.

The options probably should be set on the hi variable that is in Http.Post. It is not a member variable and you cannot set options on it with a member function. Copy Http.Post and edit.

After

__HInternet hi=HttpOpenRequest(m_hi "POST" action 0 0 0 INTERNET_FLAG_RELOAD|inetflags 0); if(!hi) ret Error

add

Member function Http.Post2
Code:
Copy      Help
int option(10) optionlen(4); if(!InternetSetOption(hi INTERNET_OPTION_MAX_CONNS_PER_SERVER &option &optionlen)) ret Error

Not tested.
#23
Can I add it in Http.PostWithTimeout?

other question:

Which you believe that it is the better option to send several Posts and receive the answer?
#24
Yes, instead of hi there is unused. But maybe the option should be set on m_hi or m_hitop. Try all.

I don't know what is the best. test all options and you will see.
#25
Ok.

I will test all options.
#26
I tried it but:

unused, m_hi ------> Error: Failed to create thread __Http_PostWithTimeout. Max 1000 threads allowed.
m_hitop -----------> Error (RT) in Http.PostWithTimeout: Exception 0xC0000005. Access violation. Cannot read memory at 0x0. In at 0x0 (0x0+0x0).

Can you modify Http.PostWithTimeout?
#27
QM does not allow more than 1000 threads.
Don't know where is the exception. Maybe because the thread was not created.
#28
Http.PostWithTimeout without line

int option(5) optionlen(4); if(!InternetSetOption(m_hi INTERNET_OPTION_MAX_CONNS_PER_SERVER &option &optionlen)) ret Error

works

note: I only run 5 Http.PostWithTimeout.
#29
I tested, works like without the line. But maybe my testing code is different.
#30
My Http.PostWithTimeout

Member function Http.PostWithTimeout
Code:
Copy      Help
function# timeout_s $action $data [str&responsepage] [$headers] [inetflags] [str&responseheaders] [&unused]

;Posts web form data. Returns 1 on success, 0 on failure. Error on timeout.
;This function cannot post files. To post files use PostFormData.

;timeout_s - timeout in seconds.
;data - urlencoded string, eg "name=John+Smith&email=j.smith@xxx.com".
;action, responsepage, headers, inetflags, responseheaders - the same as with PostFormData.
;unused - used internally.

;See also: <Http.PostFormData>.


if(!&unused)
,if(!m_hi) end ES_INIT
,__HInternet _hi; int th _ret
,th=mac("__Http_PostWithTimeout" "" &this action data &responsepage headers inetflags &responseheaders &_hi &_ret)
,wait timeout_s H th
,err
,,InternetCloseHandle _hi; _hi.handle=0 ;;ends thread
,,wait 0 H th ;;because the thread uses this. ~0.5ms
,,end _error
,ret _ret

lpstr sh="Content-Type: application/x-www-form-urlencoded"
if(empty(headers)) headers=sh
else if(findrx(headers "(?i)^Content-Type *:" 8)<0) headers=_s.from(sh "[]" headers)

unused=HttpOpenRequest(m_hi "POST" action 0 0 0 INTERNET_FLAG_RELOAD|inetflags 0); if(!unused) ret Error
int option(5) optionlen(4); if(!InternetSetOption(m_hi INTERNET_OPTION_MAX_CONNS_PER_SERVER &option &optionlen)) ret Error

if(!HttpSendRequest(unused headers -1 data len(data))) ret Error

if(&responseheaders and !GetResponseHeaders(unused responseheaders)) ret Error

if(&responsepage) ret Read(unused responsepage)
ret 1


Forum Jump:


Users browsing this thread: 1 Guest(s)