Only partial data sent to process.stdout when output reaches a certain size

[baseten]

• NPM version: 3.10.8
• Node version: v6.9.1
• Node Process:

{ http_parser: '2.7.0',
  node: '6.9.1',
  v8: '5.1.281.84',
  uv: '1.9.1',
  zlib: '1.2.8',
  ares: '1.10.1-DEV',
  icu: '57.1',
  modules: '48',
  openssl: '1.0.2j' }

• Node Platform: darwin:
• Node architecture: x64
• node-sass version:

node-sass	3.13.0	(Wrapper)	[JavaScript]
libsass  	3.3.6	(Sass Compiler)	[C/C++]

• npm node-sass versions: node-sass@3.13.0

When piping output to another script, if the compiled code is over a certain length then only the first chunk appears to be sent. From the command line all seems fine:

./node_modules/node-sass/bin/node-sass sass/app.scss

But when piping to another node script, the output is cut off:

./node_modules/node-sass/bin/node-sass sass/app.scss | scripts/test.js

Where scripts/test.js is the following:

#!/usr/bin/env node

process.stdin.on( 'readable', () => {
    var chunk = process.stdin.read();

    if ( chunk !== null ) {
        console.log( chunk.toString() ); // logs incomplete css
    }
} );

process.stdin.on( 'end', () => {
    console.log( 'end' );
} );

[baseten]

I'm not sure if this is any use, but Buffer.byteLength( chunk ) returns 65536 for my particular project. Not sure if this is related to the size of a buffer chunk or not?

[xzyfer]

My gut feeling is that this is probably either the max size of the node buffer, and the C sass string data type. We'll need to do some investigation.

…

On 8 Dec 2016 12:51 AM, "Alex Prokop" ***@***.***> wrote: I'm not sure if this is any use, but if I output the partial string to file rather than logging to console the filesize is 65,536 bytes — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1823 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAjZWG9EloRqvhK45O1_PXVOX5ZRgeMVks5rFrnfgaJpZM4LGjqD> .

[marcusdarmstrong]

I ran into this issue a few days ago. Downgraded to 3.8.0 to eliminate the blockage, looked like it was related to a node.js change to no longer flush output buffers on exit unless explicitly told to.

I was also able to workaround this by inserting this snippet into the package as a local mod:

[process.stdout, process.stderr].forEach(function(stream) {
    if (stream._handle && stream._handle.setBlocking) {
        stream._handle.setBlocking(true);
    }
});

[xzyfer]

There were two separate issues in play. Firstly console.log has max buffer length that could be exceeded by some large CSS output, and we were exiting before fully flushing the output buffer.

This has been fixed #1834.

[arunstan]

For anyone who is looking, this was shipped in v4.1.0