Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: Idhttp.Post not returning data


This question is answered. Helpful answers available: 2. Correct answers available: 1.


Permlink Replies: 30 - Last Post: Feb 22, 2016 11:57 PM Last Post By: franck clem
franck clem

Posts: 26
Registered: 9/9/01
Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 1:23 AM
Dear all,

Here is my test code

procedure TForm1.Button1Click(Sender: TObject);
var
  lst,lstReponse : TStringList;
begin
  lst := TStringList.Create;
  lstReponse := TStringList.create;
  try
    lst.add('MyVar1=toto');
    lst.add('MyVar2=titi');
    lstreponse.text := IdHttp1.Post('http://www.MyURL.com/test.php?',lst);
    showmessage(lstreponse.text); // display nothing
  finally
    lst.Free;
  end;
end;


and the test.php file:

<?php
  echo "YES !!!";
  sleep(1);
?>


nothing is returning !!

Thanks for your help

Edited by: franck clem on Feb 10, 2016 10:23 AM
Robert Triest

Posts: 687
Registered: 3/24/05
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 1:45 AM   in response to: franck clem in response to: franck clem
With exactly the same code it works fine...
(Don't forget to free your lstreponse)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 4:24 AM   in response to: Robert Triest in response to: Robert Triest
Robert Triest wrote:
With exactly the same code it works fine...
(Don't forget to free your lstreponse)

Thanks for having tested my code but I don't know why it doesn't work for me.

I've disabled my antivirus and the firewall in windows 10 and it is the same.

could you try with this url ?
htttp://www.crystalxe.com/test.php

Many thanks.
Franck

Edited by: franck clem on Feb 10, 2016 1:25 PM
Robert Triest

Posts: 687
Registered: 3/24/05
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 5:02 AM   in response to: franck clem in response to: franck clem
It doesn't work. Connection Timeout. There must be a php problem on the server side. Maybe there you are not allowed to execute php scripts..

btw..

Do you know that php has now a build in Server?
(Don't worry.. It will NOT be installed as a webserver on your machine, you just install/extract the php engine (zip) file in a directory..)

Just download latest php from the official site and extract the zip in a directory on your local disk. (like c:\php)
Create a directory where you put your php scripts, like c:\php\site and put your test.php in there.
On the command prompt you go to the "site" directory (cd c:\php\site) and start php in servermode (c:\php\php -S 0.0.0.0:8080)

(Maybe you need to install the Visual C++ driver: https://www.microsoft.com/en-us/download/details.aspx?id=48145)

In your browser you do : http://localhost:8080/test.php
In the Delphi App you do: lstreponse.text := IdHttp1.Post('http://localhost:8080/test.php?',lst);
Robert Triest

Posts: 687
Registered: 3/24/05
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 5:07 AM   in response to: franck clem in response to: franck clem
Oh.. Your ?? software looks cool. All developed in Delphi?
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 5:29 AM   in response to: Robert Triest in response to: Robert Triest
Thank for your advices.

I've made a mistake in the URL (http and not httttp) so if you had copy/paste the URL that's why you have had a time out.
The script is working because if you test it with a browser it will display "YES"
click here to test it: [http://www.crystalxe.com/test.php]

About your question, yes I've developped this software with Delphi. I've also developped a script compiler for user who want to write its own automation or animation.

Franck
Robert Triest

Posts: 687
Registered: 3/24/05
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 5:45 AM   in response to: franck clem in response to: franck clem
I've made a mistake in the URL (http and not httttp) so if you had copy/paste the URL that's why you have had a time out.
I saw this already but still I can't execute it.. The browser is doing fine but the IdHttp1 gives me a timeout..
If I execute the script with the local php server the IdHttp1 works fine.

click here to test it: [http://www.crystalxe.com/test.php]
Yes this works..

About your question, yes I've developped this software with Delphi.
I've also developped a script compiler for user who want to write its own automation or animation.
Nice, I did something similar but only with using Python as script language..
https://github.com/pyscripter/python4delphi
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 6:07 AM   in response to: Robert Triest in response to: Robert Triest
nice work !!

I'dont know why you have a timeout, but this should work.
I've deleted the sleep in the php script.

With Get I don't have any problem, and I think you too.

procedure TForm1.Button2Click(Sender: TObject);
begin
  try
    showmessage(IdHttp1.Get('http://www.crystalxe.com/test.php'));
  finally
  end;
end;


So my problem is not solved and I don't know how to post data...
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 8:42 AM   in response to: franck clem in response to: franck clem
I've installed a WAMP server on another computer on my local network and it works fine, I can display the echo of the test.php file.
So I think there is a limitation with the web hosting but there is nothing to configure in the dashboard. (OVH)
Robert Triest

Posts: 687
Registered: 3/24/05
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 12:43 PM   in response to: franck clem in response to: franck clem
So I think there is a limitation with the web hosting but there is nothing to configure in the dashboard. (OVH)
Maybe a setting on webserver or php.ini?

http://php.net/manual/en/faq.installation.php#faq.installation.requestmethods
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 12:57 PM   in response to: franck clem in response to: franck clem
franck wrote:

Here is my test code
<snip>
nothing is returning !!

Works fine for me using the latest SVN revision of Indy 10 (using the real
crystalxe.com URL). The ShowMessage() displays 'YES !!!' as expected, using
both TIdHTTP.Post() and TIdHTTP.Get().

--
Remy Lebeau (TeamB)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 1:12 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
I think I have found where is the problem because I've tried at home and it works.
So I think it is the firewall of my company.

But now I've to find how to by-pass it.

Franck
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 10, 2016 2:38 PM   in response to: franck clem in response to: franck clem
franck wrote:

I think I have found where is the problem because I've tried at
home and it works. So I think it is the firewall of my company.

But now I've to find how to by-pass it.

Firewalls block connections, not data content. If Post() successfully connects
to the server, a firewall can't blank out the data being received. So something
else has to be happening. Use a packet sniffer, like Wireshak, or attach
one of Indy's TIdLog... components to TIdHTTP, to capture the raw data actually
being received by TIdHTTP, make sure the server is not returning blank data
unexpectedly. For Post() to return a blank string and not throw an exception,
either the response is blank data to begin with, or the lower-level RTL is
not able to decode the data and is returning a blank conversion.

--
Remy Lebeau (TeamB)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 11, 2016 1:55 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
I've done a lot of tests and I can confirm the network block the returned data.

1) if I connect my office computer to the hotspot of my mobile phone, then it works, if displays "YES!!"
2) if I change the php code to this:

<?php
  $post  = 'POST=';
  $post .= print_r($_POST,true);
  $post .= ' GET=';
  $post .= print_r($_GET,true);
  header('Content-Description: File Transfer');
  header("Content-disposition: attachment; filename={$file}"); //Tell the filename to the browser
  header('Expires: 0');
  header('Content-type: application/octet-stream'); //Stream as a binary file! So it would force browser to download
  header('Cache-Control: must-revalidate');
  header('Pragma: public');
  header('Content-Length: '.strlen($post));
  print $post;
?>

This works also and I can display the data of the GET and the POST with this code:

procedure TForm1.BtnTestDLClick(Sender: TObject);
Var
  params : TStringList;
  res : TStringStream;
begin
  params := TStringList.Create;
  params.Add('config1='+'content1');
  params.Add('config2='+'content2');
  res := TStringStream.Create;
  idhttp1.Post('http://www.crystalxe.com/test_dl.php?var1=testget1&var2=testget2',params,res);
 
  ShowMessage(res.DataString);
 
  res.Free;
  params.free;
end;


This displays:
POST=Array
(
    [config1] => content1
    [config2] => content2
)
 GET=Array
(
    [var1] => testget1
    [var2] => testget2
)

So this method works but it is not a good way to do that (need to know the length to send..)

3) I have changed the property Request.useragent to be the same as firefox, and AcceptLanguage to en-US but it doesn't work, nothing is displayed.

So I don't know why data are always not returning for a standard post.
I'll try to install wireshark to analyse the frames.

thanks
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 11, 2016 6:26 AM   in response to: franck clem in response to: franck clem
Here is the analysis with wireshark

I've tested with a basic html file which contain a form and a textarea with the action="http://www.crystalxe.com/test.php"
Then surprise ! the browser uses the GET function to post the data. I've tried with Firefox, opera, chrome and internet explorer, and all use the GET function.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 11, 2016 2:17 PM   in response to: franck clem in response to: franck clem
franck wrote:

Then surprise ! the browser uses the GET function to post the data.
I've tried with Firefox, opera, chrome and internet explorer, and all
use the GET function.

As they should be. An HTML webform is submitted using GET unless it has
a "method=post" attribute on it, then it is submitted using POST instead.

--
Remy Lebeau (TeamB)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 12, 2016 12:50 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
But did you verify that that the data is actually being transmitted? Did
you use a packet sniffer/logger, like I suggested?

Yes data is being transmitted, I 've checked with wireshark, and I have already mofify the php code to save the post data to a text file and I can see all my data in the text file.

As they should be. An HTML webform is submitted using GET unless it has
a "method=post" attribute on it, then it is submitted using POST instead.

oh yes, I've forget to specify the method=post, that's true.

<html>
<body>
<form action="http://www.crystalxe.com/test.php">
<textarea name="config" rows="20" cols="50">content</textarea><br>
<input type="submit" value="Submit">
</form>
</body>
</html>

I'll try again with post method
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 12, 2016 1:28 AM   in response to: franck clem in response to: franck clem
I've tried with method=post and I can see the answer (="YES!!") in firefox.

So the problem is to know how to configure idHTTP to work like firefox.

In wireshark, I can see the packet HTTP POST and just after I see the answer with the text "YES !!" at the end of a HTTP frame

This is the printable text only of the frame sent by firefox:
2\E@!yPAtPPOST /test.php HTTP/1.1
Host: www.crystalxe.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 Firefox/44.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Cookie: mediaplanBAK=R129289740; mediaplan=R3721135442
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 14
config=content

And this is the printable text only of the frame sent by IdHttp but in the next frame there is no answer, so the text "YES!!" is not in the received frame (blocked by the local network?):
2\E'3@!P44EPPOST /test.php HTTP/1.0
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 11
Host: www.crystalxe.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: identity
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0
Cookie: mediaplan=R3721135442; mediaplanBAK=R129289740

config=test
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 12, 2016 9:26 AM   in response to: franck clem in response to: franck clem
franck wrote:

This is the printable text only of the frame sent by firefox:

And this is the printable text only of the frame sent by IdHttp

The two requests are functionally equivilent. What is more important is
the response the server is sending back, but you have not shown that yet.

but in the next frame there is no answer

Well, then the server is not actually transmitting a valid response back.
Tha is a server issue, not a TIdHTTP issue.

If I had to guess, try changing the TIdHTTP.Request.UserAgent property to
match what Firefox sends. Maybe the server UserAgent-sensitive (some servers
are), not recognizing the UserAgent you are using so it doesn't send the
expected response back.

--
Remy Lebeau (TeamB)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 15, 2016 2:39 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
The two requests are functionally equivilent. What is more important is
the response the server is sending back, but you have not shown that yet.

Ok but if the response is different (betwwen firefox and IDHTTP), this is because the request is different too.

If I had to guess, try changing the TIdHTTP.Request.UserAgent property to
match what Firefox sends. Maybe the server UserAgent-sensitive (some servers
are), not recognizing the UserAgent you are using so it doesn't send the
expected response back.

I have already changed the UserAgent: "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0"

Here is screen copy and frames capture with firefox: http://www.crystalxe.com/tmp/capture_firefox/
Here is screen copy and frames capture with IDHTTP: http://www.crystalxe.com/tmp/capture_IDHTTP/

The following method works and doesn't use Indy (thank you ShaiLeTroll, a french contributor on the french forum developez.net)

procedure T1Form1.btn1Click(Sender: TObject);
const
  BufferSize = 1024;
  accept: packed array[0..1] of LPWSTR = (PChar('text/*'), nil); // PCTSTR rgpszAcceptTypes[] = {_T(“text/*”), NULL};
var
    hSession, hHTTP, hReq : HINTERNET;
    Buffer: array[1..BufferSize] of Byte;
    BufferLen: DWORD;
    f: File;
    sAppName: string;
    FS:TStringStream;
    Headers: string;
    Data: RawByteString;
begin
 
    sAppName := ExtractFileName(Application.ExeName);
 
    hSession := InternetOpen(PChar(sAppName), INTERNET_OPEN_TYPE_PRECONFIG, nil, nil, 0);
    try
      hHTTP := InternetConnect(hSession, 'www.crystalxe.com', INTERNET_DEFAULT_HTTP_PORT, nil, nil, INTERNET_SERVICE_HTTP, 0, 1);
      try
        hReq := HttpOpenRequest(hHTTP, PChar('POST'), 'test.php', nil, nil, @accept, 0, 1);
        try
          Headers := 'Content-Type: application/x-www-form-urlencoded; charset=utf-8';
          Data := UTF8Encode('param1=Value1&param2=value2&param3=Value3');
          if HttpSendRequest(hReq, PChar(Headers), length(Headers), PAnsiChar(Data), length(Data)) then
          begin
            FS := TStringStream.Create();
            try
              BufferLen := 0;
              repeat
                if InternetReadFile(hReq, @Buffer, BufferSize, BufferLen) then
                  FS.WriteBuffer(Buffer, BufferLen);
 
              until BufferLen = 0;
 
                         ShowMessage(FS.DataString); // YES !!!
            finally
              FS.Free;
            end;
          end
          else
            raise Exception.Create('HttpOpenRequest failed. ' + SysErrorMessage(GetLastError));
        finally
          InternetCloseHandle(hReq);
        end;
      finally
        InternetCloseHandle(hHTTP);
      end;
    finally
      InternetCloseHandle(hSession);
    end;
 end;

Franck

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit] [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 16, 2016 11:39 AM   in response to: franck clem in response to: franck clem
franck wrote:

Ok but if the response is different (betwwen firefox and IDHTTP), this
is because the request is different too.

Yes, and in this case there are three key differences between FireFox's request
and TIdHTTP's request that could affect the server's behavior in this situation:

1. they have different values in the User-Agent request header

2. FireFox is sending "config=content" as the request content, but your TIdHTTP
code is sending "action=connection" instead.

3. FireFox is sending cookies, TIdHTTP is not.

I have already changed the UserAgent: "Mozilla/5.0 (Windows NT 10.0;
WOW64; rv:44.0) Gecko/20100101 Firefox/44.0"

But that is not EXACTLY what FireFox uses for its User-Agent. Did you try
matching it EXACTLY?

Here is screen copy and frames capture with firefox:
http://www.crystalxe.com/tmp/capture_firefox/

Here is screen copy and frames capture with IDHTTP:
http://www.crystalxe.com/tmp/capture_IDHTTP/

It is clear from the second capture that the server is NOT sending a blank
response back to TIdHTTP. That is why TIdHTTP is returning a blank string
to your code. There is no Content-Length or Transfer-Encoding header present
in the response, and the Connection header is "close" instead of "keep-alive".
So the server is intentionally closing the socket connection and not sending
any data back. So this is a server issue, not a TIdHTTP issue.

Try this:

procedure TForm1.Button1Click(Sender: TObject);
var
  lst: TStringList;
  sReponse: string;
begin
  // if you are using an up-to-date version of Indy, you can set this
  // global variable one time instead of setting the Request.UserAgent
  // property on each request...
  //
  // IdHTTPHeaderInfo.GIdDefaultUserAgent := 'Mozilla/5.0 (Windows NT 6.1; 
rv:44.0) Gecko/20100101 Firefox/44.0';
 
  // request HTML webform, get cookies
  IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
  IdHttp1.Get('http://www.crystalxe.com/test.php');
 
  // now post the webform data...
  lst := TStringList.Create;
  try
    lst.add('config=content');
    IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
    sReponse := IdHttp1.Post('http://www.crystalxe.com/test.php', lst);
    ShowMessage(sReponse);
  finally
    lst.Free;
  end;
end;


--
Remy Lebeau (TeamB)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit] [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 12:57 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
1. they have different values in the User-Agent request header

I made "copy/paste" for the User-Agent of firefox.
To capture the user agent, I've created a special php script to display it.

2. FireFox is sending "config=content" as the request content, but your TIdHTTP
code is sending "action=connection" instead.

Yes but this in the script I don't use the parameters, the script always returns "YES !!!"

So this is a server issue, not a TIdHTTP issue.

The server runs Apache, and the hosting is OVH one of the most popular web hosting, so do you think there is a bug in Apache ? not sure.

Try this:

procedure TForm1.Button1Click(Sender: TObject);
var
  lst: TStringList;
  sReponse: string;
begin
  // if you are using an up-to-date version of Indy, you can set this
  // global variable one time instead of setting the Request.UserAgent
  // property on each request...
  //
  // IdHTTPHeaderInfo.GIdDefaultUserAgent := 'Mozilla/5.0 (Windows NT 6.1; 
rv:44.0) Gecko/20100101 Firefox/44.0';
 
  // request HTML webform, get cookies
  IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
  IdHttp1.Get('http://www.crystalxe.com/test.php');
 
  // now post the webform data...
  lst := TStringList.Create;
  try
    lst.add('config=content');
    IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
    sReponse := IdHttp1.Post('http://www.crystalxe.com/test.php', lst);
    ShowMessage(sReponse);
  finally
    lst.Free;
  end;
end;

Nothing is returned.

I've have also made a test in Java and it is working, It displays the returned string without error: "YES !!"
I've used this java code: http://www.mkyong.com/java/how-to-send-http-request-getpost-in-java/ and I've just replace the URL and "https" by "http"

So to summarize:

- It is working with web browser Opera, IE, Firewox and Google chrome.
- It is working with Java
- It is working with httpOpenRequest
- but It is not working with IdHTTP

Thanks for you help
Franck

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit] [Edit] [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 18, 2016 10:06 AM   in response to: franck clem in response to: franck clem
franck wrote:

Nothing is returned.

I don't know what else to tell you then. It works perfectly fine for me
when I test it using the same code I have shown here. Your Wireshark capture
clearly shows the server's response is missing body data, which is why
TIdHTTP would return a blank string. That is a server-side issue, or maybe
a firewall issue, but it is certainly not a TIdHTTP issue.

This code work fine for me:

var
  lst, lstReponse : TStringList;
begin
  lst := TStringList.Create;
  lstReponse := TStringList.create;
  try
    lst.add('MyVar1=toto');
    lst.add('MyVar2=titi');
    lstreponse.text := IdHttp1.Get('http://www.crystalxe.com/test.php?');
    showmessage(lstreponse.text);
  finally
    lst.Free;
    lstReponse.Free;
  end;
end;


This code works fine for me:

var
  lst: TStringList;
  sReponse: string;
begin
  // if you are using an up-to-date version of Indy, you can set this
  // global variable one time instead of setting the Request.UserAgent
  // property on each request...
  //
  // IdHTTPHeaderInfo.GIdDefaultUserAgent := 'Mozilla/5.0 (Windows NT 6.1; 
rv:44.0) Gecko/20100101 Firefox/44.0';
 
  // request HTML webform, get cookies
  IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
  IdHttp1.Get('http://www.crystalxe.com/test.php');
 
  // now post the webform data...
  lst := TStringList.Create;
  try
    lst.add('config=content');
    IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
    sReponse := IdHttp1.Post('http://www.crystalxe.com/test.php', lst);
    ShowMessage(sReponse);
  finally
    lst.Free;
  end;
end;


They both show 'YES !!!' as expected.

- but It is not working with IdHTTP

Works fine for me. Something is going wrong with your network setup somewhere.

--
Remy Lebeau (TeamB)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit] [Edit] [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 19, 2016 7:17 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
It works !!
I've added the option hoKeepOrigProtocol.
here is the code
procedure TForm1.Button1Click(Sender: TObject);
Var
  SL : Tstringlist;
  res : TStringStream;
begin
   IdHttp1.ReadTimeout := 3000;
   IdHttp1.ConnectTimeout := 2000;
   try
     SL := Tstringlist.create;
     SL.add('config=content');
     res := TStringStream.Create('', TEncoding.UTF8);
 
     IdHttp1.HTTPoptions := IdHttp1.HTTPoptions + [hoKeepOrigProtocol];
     IdHttp1.post('http://www.crystalxe.com/test.php',SL,res);
 
     showMessage('Response ='+IdHttp1.response.ResponseText); // Response= HTTP/1.1 200 OK
     showMessage('Msg size= '+IntToStr(res.Size)+' - Msg content= '+res.DataString); // Msg size= 9 - Msg content=  YES !!!
   except
   end;
 
   SL.free;
   res.Free;
end;


So it was only a problem of protocol version.

Many thanks

Franck
Robert Triest

Posts: 687
Registered: 3/24/05
Re: Idhttp.Post not returning data [Edit] [Edit] [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 19, 2016 7:31 AM   in response to: franck clem in response to: franck clem
Amen..
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 19, 2016 9:32 AM   in response to: franck clem in response to: franck clem
franck wrote:

It works !!
I've added the option hoKeepOrigProtocol.

That is not necessary. And as you can see in the examples I have posted
so far, I am not using the hoKeepOrigProtocol flag and yet the code still
works fine.

So it was only a problem of protocol version.

No, it is not, actually.

Using this exact code:

var
  lst, lstReponse : TStringList;
begin
  lst := TStringList.Create;
  lstReponse := TStringList.create;
  try
    lst.add('MyVar1=toto');
    lst.add('MyVar2=titi');
    lstreponse.text := IdHttp1.Get('http://www.crystalxe.com/test.php?');
    showmessage(lstreponse.text);
  finally
    lst.Free;
    lstReponse.Free;
  end;
end;


Here is the actual HTTP request and response:

GET /test.php HTTP/1.1
Host: www.crystalxe.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/3.0 (compatible; Indy Library)

HTTP/1.1 200 OK
Set-Cookie: mediaplanBAK=R129291918; path=/; expires=Fri, 19-Feb-2016 18:33:13
GMT
Date: Fri, 19 Feb 2016 17:15:01 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: mediaplan=R2045055074; path=/; expires=Fri, 19-Feb-2016 18:31:54
GMT
Server: Apache
X-Powered-By: PHP/5.6.17
Vary: Accept-Encoding

9
YES !!!
0

Using this exact code:

var
  lst: TStringList;
  sReponse: string;
begin
  // request HTML webform, get cookies
  IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
  IdHttp1.Get('http://www.crystalxe.com/test.php');
 
  // now post the webform data...
  lst := TStringList.Create;
  try
    lst.add('config=content');
    IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 
Firefox/44.0';
    sReponse := IdHttp1.Post('http://www.crystalxe.com/test.php', lst);
    ShowMessage(sReponse);
  finally
    lst.Free;
  end;
end;


Here are the actual HTTP requests and responses:

GET /test.php HTTP/1.1
Host: www.crystalxe.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 Firefox/44.0

HTTP/1.1 200 OK
Set-Cookie: mediaplanBAK=R129291918; path=/; expires=Fri, 19-Feb-2016 18:31:32
GMT
Content-Type: text/html; charset=UTF-8
Set-Cookie: mediaplan=R2045055074; path=/; expires=Fri, 19-Feb-2016 18:39:17
GMT
Server: Apache
Vary: Accept-Encoding
Transfer-Encoding: chunked
Date: Fri, 19 Feb 2016 17:18:53 GMT
Connection: keep-alive
X-Geo: varn13.rbx5
X-Geo-Port: 1006
X-Cacheable: Not cacheable: no cache headers from backend

009
YES !!!
0

POST /test.php HTTP/1.0
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 14
Host: www.crystalxe.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 Firefox/44.0
Cookie: mediaplanBAK=R129291918; mediaplan=R2045055074

config=content

HTTP/1.1 200 OK
Set-Cookie: mediaplan=R2045055074; path=/; expires=Fri, 19-Feb-2016 18:31:33
GMT
Server: Apache
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8
Date: Fri, 19 Feb 2016 17:18:53 GMT
Connection: close
X-Geo: varn13.rbx5
X-Geo-Port: 1006
X-Cacheable: Not cacheable: no cache headers from backend

YES !!!

As you can see, 'YES !!!' is always present regardless of the protocol version
being used.

--
Remy Lebeau (TeamB)

franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 22, 2016 12:17 AM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
As you can see, 'YES !!!' is always present regardless of the protocol version
being used.

Yes, sure, it is the same for me when I run this test at home.
But not at work !! the problem should be the firewall.

I have tried with your code and I've added a checkbox to enable or not the option

procedure TForm1.Button4Click(Sender: TObject);
var
  lst: TStringList;
  sReponse: string;
begin
  // request HTML webform, get cookies
  IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 Firefox/44.0';
  IdHttp1.Get('http://www.crystalxe.com/test.php');
 
  // now post the webform data...
  lst := TStringList.Create;
  try
    lst.add('config=content');
    IdHttp1.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 Firefox/44.0';
 
    if Checkbox1.Checked
        then IdHttp1.HTTPoptions := IdHttp1.HTTPoptions + [hoKeepOrigProtocol]
        else IdHttp1.HTTPoptions := IdHttp1.HTTPoptions - [hoKeepOrigProtocol];
 
    sReponse := IdHttp1.Post('http://www.crystalxe.com/test.php', lst);
    ShowMessage(sReponse);
  finally
    lst.Free;
  end;
end;


If I check the box, then it displays "YES !!"
If I don't check the box, then it displays an empty message.

At home, or by using a hotspot wifi with my mobile phone, in both case, it always display "YES !!'

For infirmation, I'm using delphi XE5.


Sending POST without option ( - [hoKeepOrigProtocol] )


2\E(@`!PiDPxPOST /test.php HTTP/1.0
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 14
Host: www.crystalxe.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Encoding: identity
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 Firefox/44.0
Cookie: mediaplan=R3757119102; mediaplanBAK=R129298452

config=content

***********************
Answer to the post
**********************
HTTP/1.1 200 OK
Set-Cookie: mediaplan=R3757119102; path=/; expires=Mon, 22-Feb-2016 09:14:59 GMT
Server: Apache
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8
Date: Mon, 22 Feb 2016 08:10:40 GMT
Connection: close

followed by 3 TCP error frames


Sending POST with option enabled ( + [hoKeepOrigProtocol] )


2\E1N@`!)PIgPxPOST /test.php HTTP/1.1
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 14
Host: www.crystalxe.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Encoding: identity
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:44.0) Gecko/20100101 Firefox/44.0
Cookie: mediaplan=R3757119102; mediaplanBAK=R129298452

config=content

****************************************
Answer to the post
****************************************
\2EJ@$!`P)gP:HTTP/1.1 200 OK
Set-Cookie: mediaplan=R3757119102; path=/; expires=Mon, 22-Feb-2016 09:13:58 GMT
Server: Apache
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Date: Mon, 22 Feb 2016 08:14:19 GMT
Connection: keep-alive

009
YES !!!
0

No frame in error

Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 22, 2016 10:45 AM   in response to: franck clem in response to: franck clem
franck wrote:

Yes, sure, it is the same for me when I run this test at home. But not
at work !! the problem should be the firewall.

The problem is either in a router/firewall/proxy that is sitting in between
your client and the server, or the problem is the server itself. If possible,
try running a packet sniffer on the server itself. If what the server is
actually sending is not the same as what the client is receiving, then the
problem is really a router/firewall.proxy getting in the way. But no matter
what, the problem is NOT IN YOUR CLIENT CODE, so you need to stop focusing
on it and look outside your app.

--
Remy Lebeau (TeamB)
franck clem

Posts: 26
Registered: 9/9/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 22, 2016 11:57 PM   in response to: Remy Lebeau (Te... in response to: Remy Lebeau (Te...
The problem is either in a router/firewall/proxy that is sitting in between
your client and the server, or the problem is the server itself. If possible,
try running a packet sniffer on the server itself. If what the server is
actually sending is not the same as what the client is receiving, then the
problem is really a router/firewall.proxy getting in the way. But no matter
what, the problem is NOT IN YOUR CLIENT CODE, so you need to stop focusing
on it and look outside your app.

Thanks a lot Remy,
Now, I know where is the problem and I have a solution which works, then I stop researches.

Best regards, and thanks again for your help.
Franck
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 12, 2016 9:19 AM   in response to: franck clem in response to: franck clem
franck wrote:

<form action="http://www.crystalxe.com/test.php">

That use GET, not POST.

I'll try again with post method

GET versus POST does not really matter. What matters is that you send the
correct request with TIdHTTP.

For GET, you have to put the data in the URL query string:

procedure TForm1.Button1Click(Sender: TObject);
var
  Reponse: string;
begin
  Reponse := IdHttp1.Get('http://www.MyURL.com/test.php?MyVar1=toto&MyVar2=titi');
  ShowMessage(Reponse);
end;


For POST, you have to put the data in the message body instead. In which
case, you are subject to the webform's "enctype" attribute. If missing,
or set to "application/x-www-form-urlencoded", post the data using a TStringList:

procedure TForm1.Button1Click(Sender: TObject);
var
  lst: TStringList;
  Reponse: string;
begin
  lst := TStringList.Create;
  try
    lst.add('MyVar1=toto');
    lst.add('MyVar2=titi');
    Reponse := IdHttp1.Post('http://www.MyURL.com/test.php', lst);
  finally
    lst.Free;
  end;
  ShowMessage(Reponse);
end;


If set to "multipart/form-data, post the data using a TIdMultipartFormDataStream
instead:

procedure TForm1.Button1Click(Sender: TObject);
var
  lst: TIdMultipartFormDataStream;
  Reponse: string;
begin
  lst := TIdMultipartFormDataStream.Create;
  try
    lst.AddFormField('MyVar1', 'toto');
    lst.AddFormField('MyVar2', 'titi');
    Reponse := IdHttp1.Post('http://www.MyURL.com/test.php', lst);
  finally
    lst.Free;
  end;
  ShowMessage(Reponse);
end;


--
Remy Lebeau (TeamB)
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Idhttp.Post not returning data [Edit] [Edit]  
Click to report abuse...   Click to reply to this thread Reply
  Posted: Feb 11, 2016 2:14 PM   in response to: franck clem in response to: franck clem
franck wrote:

So this method works but it is not a good way to do that (need to know
the length to send..)

Unless you send a 'chunked' response, which is designed for responses that
do not know the length ahead of time. All HTTP 1.1+ clients and servers
are required to support chunking. See RFC 2616 Section 3.6 "Transfer Codings"
for more details:

https://tools.ietf.org/html/rfc2616#section-3.6

3) I have changed the property Request.useragent to be the same as
firefox, and AcceptLanguage to en-US but it doesn't work, nothing is
displayed.

But did you verify that that the data is actually being transmitted? Did
you use a packet sniffer/logger, like I suggested?

So I don't know why data are always not returning for a standard post.
I'll try to install wireshark to analyse the frames.

Or, you could just use Indy's TIdLog... components, like I mentioned earlier.

--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02