Login A Site With Curl Via POST

I Need Log In A Website via curl and get the index of the page.

    <form method="post">
cURL Test
Please Enter Username and Password/div>
  • UserName
  • Password
</form>

It is the website code. How i login this website using curl? I searched it and found a curl script but it doesnt work.

Source: stackoverflow-php

classes which have dependencies other other classes

I’m looking for some direction regarding the following, I’m new to OOP and getting there but think either my lack of understanding is causing me to get stuck in a rabbit hole or I’m just over thinking things too much and being anal.

basically i have a main class called “CurlRequest” which sole purpose is to perform curl requests, providing a url and params it returns me some html. This class works and functions as intended and I’m happy with that.

I use this class for a few projects but for one I then wanted to track the performance of my requests made. attempted, failed, passed etc, so i created a static class for this which manages all my counters. I place counter references like the following at different areas in my CurlRequest class.

PerformanceTracker::Increment('CurlRequest.Attempted');
PerformanceTracker::Increment('CurlRequest.Passed');
PerformanceTracker::Increment('CurlRequest.Failed');

I have around 10 or so of these with my class tracking all kinds of things during the curl request and i also use my PerformanceTracker class in other classes i made.

However like mentioned i only wanted to do this for one of my projects, so find my self in the situation of having my original CurlRequest class and an altered one with performance counters in it.

My question is, is their a way i can use the same class for any project and choose to use the PerformanceTracker class or not. The obvious way i thought of was to pass an $option argument into the class and then have if statements around all the counters, but can’t help think its messy.

            if ($this->options['perfCounter'] == true ) {

                PerformanceTracker::Increment($this->owner . '.CurlRequest.Failed');

            }

this also adds a lot of extra code to the class.


Source: stackoverflow-php

How I can get specific picture on a website using curl xpath in php

Through this code I can get the specific text but my problem is how I can get the specific picture on a website through this code. Can anyone help, I am stuck at this point.

<?php

ini_set('max_execution_time', 300);

function news($url,$path){

    $curl=curl_init($url);

    curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);

    curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false);

    $html=curl_exec($curl);

    if(!empty($curl)){

        $thispage= new DOMDocument;
        libxml_use_internal_errors(true);
        $thispage->loadHTML($html);
        libxml_clear_errors();
        $xpath=new DOMXPath($thispage);
        $status=$xpath->evaluate($path);
        return $status;
    }
}

$a= news('https://www.dawn.com/latest-news','string(/html/body/div[2]/div/main/div/div/div[1]/article[1]/h2/a)');

echo $a;

?>


Source: stackoverflow-php

Translating PHP http request into C# .NET

I have written this code in PHP to make a HTTP request to my server to retrieve my data:

<?php


$client_secret= '';
$data= array(

'email' => '**********',

'password' => '******',

'client_id' => '*******'
);

$api_url='******';

$json_data=json_encode($data);

$signature_string = md5($json_data . $client_secret); 


$post_data = 'signature='.$signature_string.'&data='.urlencode($json_data);

$curl = curl_init($api_url); 
curl_setopt($curl, CURLOPT_SSL_VERIFYHOST, 1);

curl_setopt($curl, CURLOPT_SSL_VERIFYPEER, false); 

curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($curl, CURLOPT_POST, 1); 

curl_setopt($curl, CURLOPT_POSTFIELDS, $post_data);

$result = curl_exec($curl); 

print_r($result);


curl_close($curl); 

?> 

And this works good. Data is returned in JSON format. Now I’m trying to do exactly the same thing but in .NET

I have managed to recreate in .NET C# exactly the same postData parameter as in PHP, but I’m not sure how to pass it in C# with the request itself… So I’ve decided to use .NET’s restsharp library to make the request like this:

  public void GetResponse()
        {
            string client_secret = "*****";

            var serializer = new JavaScriptSerializer();
            string json_data = serializer.Serialize(new { email = "*****", password = "*****", client_id = "*****" });

            var signature_string = CalculateMD5Hash(json_data + client_secret).ToLower();

            var postData = "signature=" + signature_string + "&data=" + Server.UrlEncode(json_data);
            var client = new RestClient("mysite.com");
            var request = new RestRequest(Method.POST);
            request.AddHeader("content-type", "application/x-www-form-urlencoded");
            request.AddHeader("cache-control", "no-cache");
            request.AddParameter("application/x-www-form-urlencoded", "signature="+signature_string, ParameterType.RequestBody);
            request.AddParameter("application/x-www-form-urlencoded", "data=" +json_data, ParameterType.RequestBody);

            IRestResponse response = client.Execute(request);

        }

Please note in C# and in PHP variables:

$post_data 

and 

var postdata

are exactly identical. Now I just don’t know how to pass them alongside the request because the server always responds me that request data was empty..

What am I doing wrong here?


Source: stackoverflow-php

php curl auto fill and login

I have created a script to auto fill and login to a website but it doesn’t work. One reason surely is that I don’t know how to submit.

That’s my code:

?php

$dir = 'C:/Users/mmea/Desktop/temporary/';

$user = "fuff";
$pass = "fuffi";

$url="https://www.website.com/sign-in/";

$postinfo = "username=".urlencode($user)."&password=".urlencode($pass)."&autologin=1&signin-submit2=Submit";

$cookie_file_path = $dir."/cookie.txt";

echo login($url, $postInfo);

function login($url,$data){

    $fp = fopen("cookie.txt", "w");
    fclose($fp);
    $login = curl_init();

    curl_setopt($login, CURLOPT_COOKIEJAR, "cookie.txt");
    curl_setopt($login, CURLOPT_COOKIEFILE, "cookie.txt");
    curl_setopt($login, CURLOPT_TIMEOUT, 40000);
    curl_setopt($login, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($login, CURLOPT_URL, $url);
    curl_setopt($login, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt($login, CURLOPT_FOLLOWLOCATION, TRUE);
    curl_setopt($login, CURLOPT_POST, TRUE);
    curl_setopt($login, CURLOPT_POSTFIELDS, $data);
    curl_setopt($login, CURLOPT_SSL_VERIFYPEER, false);

    ob_start();
    curl_exec ($login);
    if(curl_errno($login)){
    throw new Exception(curl_error($login));
    }

    return curl_exec ($login);
    ob_end_clean();
    curl_close ($login);
    unset($login);    
}

?>

The username and password aren’t real.


Source: stackoverflow-php

Scaling curl with php and cron

I am trying to create a website monitoring webapp using PHP. At the minute I’m using curl to collect headers from different websites and update a MySQL database when a website’s status changes (e.g. if a site that was ‘up’ goes ‘down’).

I’m using curl_multi (via the Rolling Curl X class which I’ve adapted slightly) to process 20 sites in parallel (which seems to give the fastest results) and CURLOPT_NOBODY to make sure only headers are collected and I’ve tried to streamline the script to make it as fast as possible.

It is working OK and I can process 40 sites in approx. 2-4 seconds. My plan has been to run the script via cron every minute… so it looks like I will be able to process about 600 websites per minute. Although this is fine at the minute it won’t be enough in the long term.

So how can I scale this? Is it possible to run multiple crons in parallel or will this run into bottle-necking issues?

Off the top of my head I was thinking that I could maybe break the database into groups of 400 and run a separate script for these groups (e.g. ids 1-400, 401-800, 801-1200 etc. could run separate scripts) so there would be no danger of database corruption. This way each script would be completed within a minute.

However it feels like this might not work since the one script running curl_multi seems to max out performance at 20 requests in parallel. So will this work or is there a better approach?


Source: stackoverflow-php

Post data to external URL using cURL

I want to use cURL to post some data to an external URL, here is the code they ask to embed, for some technical needs I need to create my own form action so I need cURL so POST those data:
This is what I am asked to embed:

<form action='https://external.com/blabla' name=715000002374001 method='POST'>  
<input type='text'name='xnQsjsdp' value='c942f375287f707b73'/>  
<input type='hidden' name='zc_gad' id='zc_gad' value=''/> 
 <input type='text'name='xmIwtLD' value=1aff1efd0a4'/>  
<input type='text' name='actionType' value='TGVhZHM='/> 
<input type='text' name='returnURL' value='http://www.blabla.de' />
</form>

I try to make it with cURL :

  $data = 'xnQsjsdp=c942f375287f707b73&xmIwtLD=TGVhZHM&actionType=TGVhZHM&returnURL&http://www.blabla.de';
  $ch = curl_init();
  curl_setopt($ch,CURLOPT_URL,'https://external.com/blabla');
  curl_setopt($ch, CURLOPT_URL,$url);
  //curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
  curl_setopt($ch, CURLOPT_POST, 1);
  curl_setopt($ch, CURLOPT_POSTFIELDS, $data );
  $res = curl_exec ($ch);
  curl_close ($ch);

But seems not to be working !!!


Source: stackoverflow-php