Use `-` as the input file:

    cat largefile.tgz.aa largefile.tgz.ab | tar zxf -

Make sure you cat them in the same order they were split.

If you&#39;re using zsh you can use the multios feature and avoid invoking cat:

    &lt; largefile.tgz.aa &lt; largefile.tgz.ab tar zxf -

Or if they are in alphabetical order:

    &lt;largefile.tgz.* | tar zxf -

Is it possible to restrict a Symfony 2 route for XHR requests only? I want to declare routes, which are only accessible via AJAX.

I do not want to put some extra lines into each AJAX-specific-actions like that:

    if ($request-&gt;isXmlHttpRequest()) {
        // do something
    } else {
        // do something else
    }

I want to define:

- one rule for AJAX requests
- one rule for GET/POST requests to the same URL

in order to get around having conditions like above.

Is it possible to restrict a route for AJAX only?

    &gt;&gt;&gt; import pytz
    &gt;&gt;&gt; pytz.timezone(&#39;Asia/Hong_Kong&#39;)
    &lt;DstTzInfo &#39;Asia/Hong_Kong&#39; LMT+7:37:00 STD&gt;

A seven hour and 37 minute offset? This is a little strange, does anyone experience the same issue?

In fact I&#39;m getting different behavior between

    import pytz
    from datetime import datetime
    hk = pytz.timezone(&#39;Asia/Hong_Kong&#39;)

    dt1 = datetime(2012,1,1,tzinfo=hk)
    dt2 = hk.localize(datetime(2012,1,1))
    if dt1 &gt; dt2:
       print &quot;Why?&quot;




Weird timezone issue with pytz

I have a large tar file I `split`. Is it possible to `cat` and untar the file using pipeline.  

Something like:

    cat largefile.tgz.aa largefile.tgz.ab | tar -xz

instead of:

    cat largefile.tgz.aa largfile.tgz.ab &gt; largefile.tgz
    tar -xzf largefile.tgz

I have been looking around and I can&#39;t find the answer. I wanted to see if it was possible.

How to extract tar archive from stdin?

I have a large tar file I <code>split</code>. Is it possible to <code>cat</code> and untar the file using pipeline.
Something like:
<pre><code class="hljs language-bash">cat largefile.tgz.aa largefile.tgz.ab | tar -xz
</code></pre>
instead of:
<pre><code class="hljs language-bash">cat largefile.tgz.aa largfile.tgz.ab > largefile.tgz
tar -xzf largefile.tgz
</code></pre>
I have been looking around and I can't find the answer. I wanted to see if it was possible.

How can I pipe the stdout of multiple commands to a single command?  

Example 1: combine and sort the output of all three echo commands:

    echo zzz; echo aaa; echo kkk
desired output:

    aaa
    kkk
    zzz


Example 2: rewrite the following so that all the commands are in a single command-line using pipes, without redirects to a temp file:

    setopt &gt; /tmp/foo; unsetopt &gt;&gt; /tmp/foo; set &gt;&gt; /tmp/foo; sort /tmp/foo

Pipe multiple commands into a single command

I want to do something like this:

    $ [mysql query that produces many lines] | php parse_STDIN.php

In `parse_STDIN.php` file I want to be able to parse my data line by line from stdin.

Reading line by line from STDIN

git 1.7.12

I want to mark all files below a given directory as assume-unchanged.

1) `git update-index --assume-unchanged dir/` gives &quot;Ignoring path.&quot;  

2) `git update-index --assume-unchanged dir/*` quickly fails because it will encounter files which are not being tracked, hence it gives &quot;fatal: Unable to mark file&quot; and quits.  

3) Try generating a list of files to mark.  `cd` into the desired directory and then run `git ls-files | tr &#39;\n&#39; &#39; &#39; | git update-index --assume-unchanged`.  This produces no error message, but it does not successfully mark the files.  The first portion of the command, `git ls-files | tr &#39;\n&#39; &#39; &#39;`, correctly produces a space delimited list of all the files I want to mark.  If I copy and paste the output of that command onto the command-line, then the `git update-index` command works.  What is not working with the pipes?

No, it is not sufficient for me to add `dir` to .gitignore.  I need these files to be in the repository, but undesired changes will be made locally that need to be ignored so that users can do pulls.  

git update-index --assume-unchanged on directory

I&#39;m trying to pass the `cat` output to curl:

    $ cat file | curl --data &#39;{&quot;title&quot;:&quot;mytitle&quot;,&quot;input&quot;:&quot;-&quot;}&#39; http://api

But `input` is literally a `-`.

use pipe for curl data

I want to use `subprocess.check_output()` with `ps -A | grep &#39;process_name&#39;`.
I tried various solutions but so far nothing worked. Can someone guide me how to do it?

How to use `subprocess` command with pipes

I am trying to spawn an external process `phantomjs` using node&#39;s `child_process` and then send information to that process after it was initialized, is that possible?

I have the following code:

    var spawn = require(&#39;child_process&#39;).spawn,
        child = spawn(&#39;phantomjs&#39;);

    child.stdin.setEncoding = &#39;utf-8&#39;;
    child.stdout.pipe(process.stdout);

    child.stdin.write(&quot;console.log(&#39;Hello from PhantomJS&#39;)&quot;);

But the only thing I got on the stdout is the initial prompt for phantomjs console.

    phantomjs&gt; 

So it seems the `child.stdin.write` is not making any effect.

I am  not sure I can send additional information to phantomjs ater the initial spawn.

thanks in advance.

Nodejs Child Process: write to stdin from an already initialised process

I have a python script parse.py, which in the script open a file, say file1, and then do something maybe print out the total number of characters. 

    filename = &#39;file1&#39;
    f = open(filename, &#39;r&#39;)
    content = f.read()
    print filename, len(content)

Right now, I am using stdout to direct the result to my output file - output
    
    python parse.py &gt;&gt; output

However, I don&#39;t want to do this file by file manually, is there a way to take care of every single file automatically? Like
    
    ls | awk &#39;{print}&#39; | python parse.py &gt;&gt; output 

Then the problem is how could I read the file name from standardin? 
or there are already some built-in functions to do the ls and those kind of work easily?

Thanks!


How to open every file in a folder

Bash allows to use: `cat &lt;(echo &quot;$FILECONTENT&quot;)`

Bash also allow to use: `while read i; do echo $i; done &lt;/etc/passwd`

to combine previous two this can be used: `echo $FILECONTENT | while read i; do echo $i; done`

The problem with last one is that it creates sub-shell and after the while loop ends variable `i` cannot be accessed any more.

My question is:

How to achieve something like this: `while read i; do echo $i; done &lt;(echo &quot;$FILECONTENT&quot;)` or in other words: How can I be sure that `i` survives while loop?

Please note that I am aware of enclosing while statement into `{}` but this does not solves the problem (imagine that you want use the while loop in function and return `i` variable)

How to pipe input to a Bash while loop and preserve variables after loop ends

I have a program that interacts with the user (acts like a shell), and I want to run it using the Python subprocess module interactively.
That means, I want the possibility to write to standard input and immediately get the output from standard output. I tried many solutions offered here, but none of them seems to work for my needs.

The code I&#39;ve written is based on *https://stackoverflow.com/questions/11457931/running-an-interactive-command-from-within-python*.

    import Queue
    import threading
    import subprocess
    def enqueue_output(out, queue):
        for line in iter(out.readline, b&#39;&#39;):
            queue.put(line)
        out.close()

    def getOutput(outQueue):
        outStr = &#39;&#39;
        try:
            while True: # Adds output from the queue until it is empty
                outStr += outQueue.get_nowait()

        except Queue.Empty:
            return outStr

    p = subprocess.Popen(&quot;./a.out&quot;, stdin=subprocess.PIPE, stout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize = 1)
    #p = subprocess.Popen(&quot;./a.out&quot;, stdin=subprocess.PIPE, stout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False, universal_newlines=True)

    outQueue = Queue()
    errQueue = Queue()

    outThread = Thread(target=enqueue_output, args=(p.stdout, outQueue))
    errThread = Thread(target=enqueue_output, args=(p.stderr, errQueue))

    outThread.daemon = True
    errThread.daemon = True

    outThread.start()
    errThread.start()

    p.stdin.write(&quot;1\n&quot;)
    p.stdin.flush()
    errors = getOutput(errQueue)
    output = getOutput(outQueue)

    p.stdin.write(&quot;5\n&quot;)
    p.stdin.flush()
    erros = getOutput(errQueue)
    output = getOutput(outQueue)

The problem is that the queue remains empty, as if there is no output.
Only if I write to standard input all the input that the program needs to execute and terminate, then I get the output (which is not what I want). For example, if I do something like:

    p.stdin.write(&quot;1\n5\n&quot;)
    errors = getOutput(errQueue)
    output = getOutput(outQueue)

Is there a way to do what I want to do?

&lt;hr/&gt;

The script will run on a Linux machine. I changed my script and deleted the universal_newlines=True + set the bufsize to 1 and flushed standard input immediately after write. Still I don&#39;t get any output.

**Second try:**

I tried this solution and it works for me:

    from subprocess import Popen, PIPE

    fw = open(&quot;tmpout&quot;, &quot;wb&quot;)
    fr = open(&quot;tmpout&quot;, &quot;r&quot;)
    p = Popen(&quot;./a.out&quot;, stdin = PIPE, stdout = fw, stderr = fw, bufsize = 1)
    p.stdin.write(&quot;1\n&quot;)
    out = fr.read()
    p.stdin.write(&quot;5\n&quot;)
    out = fr.read()
    fw.close()
    fr.close()


Interactive input/output using Python

I created .tgz file with `tar czvf file ` command.then I ended up with a tgz file. I want to know the difference between it and tar.gz.

Are tar.gz and tgz the same thing?

I normally compress using `tar zcvf` and decompress using `tar zxvf` (using gzip due to habit). 

I&#39;ve recently gotten a quad core CPU with hyperthreading, so I have 8 logical cores, and I notice that many of the cores are unused during compression/decompression. 

Is there any way I can utilize the unused cores to make it faster?

Utilizing multi core for tar+gzip/bzip compression/decompression

I have a tar.gz-file with the following structure:

    folder1/img.gif
    folder2/img2.gif
    folder3/img3.gif

I want to extract the image files without the folder hierarchy so the extracted result looks like:

    /img.gif
    /img2.gif
    /img3.gif

I need to do this with a combination of Unix and PHP. Here is what I have so far, it works to extract them to the specified directory but keeps the folder hierarchy:

    exec(&#39;gtar --keep-newer-files -xzf images.tgz -C /home/user/public_html/images/&#39;,$ret);


How do I extract files without folder structure using tar

I want to extract an archive named `filename.tar.gz`.

Using `tar -xzvf filename.tar.gz` doesn&#39;t extract the file. it is gives this error:

    gzip: stdin: not in gzip format
    tar: Child returned status 1
    tar: Error exit delayed from previous errors

How to extract filename.tar.gz file

I have an archive 

&gt; *.tar.gz

How can I uncompress this in a destination directory?

How to uncompress a tar.gz in another directory

In R (thanks to [magrittr][1]) you can now perform operations with a more functional piping syntax via `%&gt;%`. This means that instead of coding this: 

    &gt; as.Date(&quot;2014-01-01&quot;)
    &gt; as.character((sqrt(12)^2)

You could also do this: 

    &gt; &quot;2014-01-01&quot; %&gt;% as.Date 
    &gt; 12 %&gt;% sqrt %&gt;% .^2 %&gt;% as.character

To me this is more readable and this extends to use cases beyond the dataframe. Does the python language have support for something similar?

[1]: https://magrittr.tidyverse.org/

Functional pipes in python like %&gt;% from R&#39;s magrittr

I typically get `PCA` loadings like this:

    pca = PCA(n_components=2)
    X_t = pca.fit(X).transform(X)
    loadings = pca.components_

If I run `PCA` using a scikit-learn pipeline:

    from sklearn.pipeline import Pipeline
    pipeline = Pipeline(steps=[    
    (&#39;scaling&#39;,StandardScaler()),
    (&#39;pca&#39;,PCA(n_components=2))
    ])
    X_t=pipeline.fit_transform(X)

is it possible to get the loadings?

Simply trying `loadings = pipeline.components_` fails:

    AttributeError: &#39;Pipeline&#39; object has no attribute &#39;components_&#39;



(Also interested in extracting attributes like `coef_` from pipelines.)

Getting model attributes from pipeline

I want to get feature names after I fit the pipeline.

    categorical_features = [&#39;brand&#39;, &#39;category_name&#39;, &#39;sub_category&#39;]
    categorical_transformer = Pipeline(steps=[
        (&#39;imputer&#39;, SimpleImputer(strategy=&#39;constant&#39;, fill_value=&#39;missing&#39;)),
        (&#39;onehot&#39;, OneHotEncoder(handle_unknown=&#39;ignore&#39;))])
        
    numeric_features = [&#39;num1&#39;, &#39;num2&#39;, &#39;num3&#39;, &#39;num4&#39;]
    numeric_transformer = Pipeline(steps=[
        (&#39;imputer&#39;, SimpleImputer(strategy=&#39;median&#39;)),
        (&#39;scaler&#39;, StandardScaler())])

    preprocessor = ColumnTransformer(
        transformers=[
            (&#39;num&#39;, numeric_transformer, numeric_features),
            (&#39;cat&#39;, categorical_transformer, categorical_features)])
Then

    clf = Pipeline(steps=[(&#39;preprocessor&#39;, preprocessor),
                          (&#39;regressor&#39;, GradientBoostingRegressor())])

After fitting with pandas dataframe, I can get feature importances from 

`clf.steps[1][1].feature_importances_`

and I tried `clf.steps[0][1].get_feature_names()` but I got an error 

    AttributeError: Transformer num (type Pipeline) does not provide get_feature_names.

How can I get feature names from this?


Content Type	Original Author	Original Content on Stackoverflow
Question	Charlie	View Question on Stackoverflow
Solution 1 - Pipe	Thor	View Answer on Stackoverflow

How to extract tar archive from stdin?

Pipe Problem Overview

Pipe Solutions

Solution 1 - Pipe

Weird timezone issue with pytz

Is it possible to restrict a route for AJAX only?

Attributions