Page.setContent should wait for resources to be loaded #728

aslushnikov · 2017-09-09T04:18:30Z

(as mentioned in #486 and other places)

We need a way to wait for page to load all the resources after the page.setContent.

The text was updated successfully, but these errors were encountered:

aslushnikov · 2017-10-04T21:56:57Z

Meanwhile. a good workaround for page.setContent that waits for all the resources to load:

await page.goto(`data:text/html,${html}`, { waitUntil: 'networkidle0' });

cmazakas · 2017-10-10T17:19:31Z

It's amazing that you posted the workaround. I'm running exactly into this issue, attempting to use puppeteer as a PDF generating service from HTML.

Thank you for filing a formal issue.

…rect html. More here puppeteer/puppeteer#728

This patch: - migrates navigation watcher to use protocol-issued lifecycle events. - removes `networkIdleTimeout` and `networkIdleInflight` options for `page.goto` method - adds a new `networkidle0` value to the waitUntil option of navigation methods References puppeteer#728. BREAKING CHANGE: As an implication of this new approach, the `networkIdleTimeout` and `networkIdleInflight` options are no longer supported. Interested clients should implement the behavior themselves using the `request` and `response` events.

This patch: - migrates navigation watcher to use protocol-issued lifecycle events. - removes `networkIdleTimeout` and `networkIdleInflight` options for `page.goto` method - adds a new `networkidle0` value to the waitUntil option of navigation methods References #728. BREAKING CHANGE: As an implication of this new approach, the `networkIdleTimeout` and `networkIdleInflight` options are no longer supported. Interested clients should implement the behavior themselves using the `request` and `response` events.

This patch adds "options" parameter to the `page.setContent` method. The parameter is the same as a navigation parameter and allows to specify maximum timeout to wait for resources to be loaded, as well as to describe events that should be emitted before the setContent operation would be considered successful. Fixes puppeteer#728.

This patch adds "options" parameter to the `page.setContent` method. The parameter is the same as a navigation parameter and allows to specify maximum timeout to wait for resources to be loaded, as well as to describe events that should be emitted before the setContent operation would be considered successful. Fixes #728.

This patch: - migrates navigation watcher to use protocol-issued lifecycle events. - removes `networkIdleTimeout` and `networkIdleInflight` options for `page.goto` method - adds a new `networkidle0` value to the waitUntil option of navigation methods References puppeteer#728. BREAKING CHANGE: As an implication of this new approach, the `networkIdleTimeout` and `networkIdleInflight` options are no longer supported. Interested clients should implement the behavior themselves using the `request` and `response` events.

…eteer#1152) This patch adds "options" parameter to the `page.setContent` method. The parameter is the same as a navigation parameter and allows to specify maximum timeout to wait for resources to be loaded, as well as to describe events that should be emitted before the setContent operation would be considered successful. Fixes puppeteer#728.

murilozilli · 2017-11-22T22:08:53Z

I'm having trouble with this too on version 0.13-alpha with browser configs as:

"waitUntil": "networkidle2",
"timeout": 60000

HanXHX · 2017-12-07T11:29:10Z

I confirm the same trouble.

Padam87 · 2017-12-13T15:52:16Z

with the latest release this hangs too:

await page.goto(`data:text/html,${html}`, { waitUntil: 'networkidle' });

wait for networkidle0 instead

await page.goto(`data:text/html,${html}`, { waitUntil: 'networkidle0' });

HanXHX · 2017-12-15T13:27:14Z

Hey @aslushnikov

You said in #1312 to wait for https://chromium-review.googlesource.com/c/chromium/src/+/747805
The patch is merged...

Everything is OK to solve this issue?

Cheers!

aslushnikov · 2018-01-10T06:19:18Z

@HanXHX this requires more work upstream: in order to reuse lifecycle events, page.setContent should initiate a navigation, which in turn should be plumbed through browser-side navigation aka "plznavigate".

tzieleniewski · 2018-10-10T11:54:55Z

Hi Team!

Please advise at the WO is not working in our case. (puppeteer 1.9.0)
I am trying to convert the XHTML content. I am providing XHTML content as an excaped inlined string.
The generated document contains raw XHTML and there are no external resources requests (in this case CSS).

Example

'use strict'

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch()
    const page = await browser.newPage()
    await page.setRequestInterception(true)

    const xhtml = `&lt;!DOCTYPE html PUBLIC &quot;-//W3C//DTD XHTML 1.0 Transitional//EN&quot; &quot;http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd&quot;&gt; &lt;html xmlns=&quot;http://www.w3.org/1999/xhtml&quot;&gt; &lt;head&gt; &lt;meta charset=utf-8&quot;/&gt; &lt;meta http-equiv=&quot;Content-Type&quot; content=&quot;application/xhtml+xml; charset=utf-8&quot;/&gt; &lt;link href=&quot;css.css&quot; media=&quot;all&quot; rel=&quot;stylesheet&quot; type=&quot;text/css&quot;/&gt; &lt;/head&gt; &lt;body&gt; some content &lt;/body&gt; &lt;/html&gt;`

    console.log(xhtml)

    page.on('request', request => {
        console.log(`Intercepted request with URL: ${request.url()}`)
        request.continue()
    });

    await page.goto(`data:text/html,${xhtml}`, {
        waitUntil: 'networkidle0'
    });
    await page.pdf({
        path: 'xhtml.pdf'
    })
    await browser.close()
})()

Here is the initial document content

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
   <head>
      <meta charset=utf-8"/>
      <meta http-equiv="Content-Type" content="application/xhtml+xml; charset=utf-8"/>
      <link href="css.css" media="all" rel="stylesheet" type="text/css"/>
   </head>
   <body>
      some content 
   </body>
</html>

ObviouslyGreen · 2018-11-16T00:24:57Z

Is there a way to set the url if we end up using the data:text/html workaround? Do relative paths for resources work via this method?

This roll includes: - https://crrev.com/608658 - DevTools: emit "init" lifecycle event when document gets opened References puppeteer#728

This roll includes: - https://crrev.com/608658 - DevTools: emit "init" lifecycle event when document gets opened References #728

cape-dev · 2018-11-20T13:52:16Z

I had the exact same problem with external resources. So the workaround from @aslushnikov helped me a lot. But as @ObviouslyGreen points out it lacks the support of resolving relative paths. I investigated what puppeteer takes as "url" when using this workaround and it is the whole html (obviously).

I could solve the problem with relative paths (for me in CSS styles) with the following approach:

create a folder (let's name it dist) in which all relative resources are placed in
generate the html as needed (paths should be relative to the root of dist)
write the html file to the root of dist
use the following code to load the html with all the relative resources resolved correctly:

const pathToHtml = path.join(__dirname, 'dist', `${randomName}.html`);

const page = await browser.newPage();
await page.goto(`file:${pathToHtml}`, { waitUntil: 'networkidle0' });

Note that the html file needs to have the '.html' suffix for puppeteer to render the html properly (at least for me this was the case).

Fixes puppeteer#728

This patch teaches `page.setContent` to await resources in the new document. **NOTE**: This patch changes behavior: currently, `page.setContent` awaits the `"domcontentloaded"` event; with this patch, we can now await other lifecycle events, and switched default to the `"load"` event. The change is justified since current behavior made `page.setContent` unusable for its main designated usecases, pushing our client to use [dataURL workaround](#728 (comment)). Fixes #728

cape-dev · 2018-11-21T07:56:31Z

@aslushnikov Great to see the same options for page.setContent!

Is it possible now with page.setContentto load resources with relative paths as I described in my workaround in the above comment?

aslushnikov · 2018-11-21T18:34:45Z

@kamekazemaster yeah, the paths should be resolved against the page's URL.

await page.goto('https://example.com');
// logo.png becomes https://example.com/logo.png
await page.setContent('<img src="/logo.png"></img>');

tzieleniewski · 2018-11-22T14:17:57Z

@aslushnikov when can we expect next release with updated setContent?

`page.goto` -> `page.setContent` per puppeteer/puppeteer#728 and https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetcontenthtml-options.

Per puppeteer/puppeteer#728 and https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetcontenthtml-options. The latest `puppeteer` seems to have broken PDF generation (with the [`page.goto`](puppeteer/puppeteer#728 (comment)) workaround), at least locally for me both here in `resume-cli` and in [`jsonresume-theme-randytarampi`](https://www.npmjs.com/package/jsonresume-theme-randytarampi).

iksheth · 2019-08-26T14:32:48Z

@kamekazemaster yeah, the paths should be resolved against the page's URL.
await page.goto('https://example.com');
// logo.png becomes https://example.com/logo.png
await page.setContent('<img src="/logo.png"></img>');

This solution is really helpful, though if described with little bit details might have saved some time.

Thank you.

casesolved-co-uk · 2024-05-16T23:18:01Z

@kamekazemaster yeah, the paths should be resolved against the page's URL.
await page.goto('https://example.com');
// logo.png becomes https://example.com/logo.png
await page.setContent('<img src="/logo.png"></img>');

Can anyone confirm if adding a <base href=...> tag to the html works instead of the initial page.goto above ?

aslushnikov self-assigned this Sep 9, 2017

aslushnikov mentioned this issue Sep 9, 2017

Reusing Browser And Page #486

Closed

aslushnikov mentioned this issue Sep 25, 2017

Waiting for both load and networkIdle #805

Closed

graingert mentioned this issue Sep 28, 2017

page.setContent(html, {waitUntil: 'networkidle'}); #907

Closed

devsnek mentioned this issue Sep 29, 2017

add page navigation load handlers to setContent #913

Closed

aslushnikov mentioned this issue Oct 4, 2017

Wait for navigation (networkidle) even if no new requests are instantiated #945

Closed

kimmobrunfeldt added a commit to alvarcarto/url-to-pdf-api that referenced this issue Oct 13, 2017

Fix issue where resource loading was not finished before rendering di…

7aee56d

…rect html. More here puppeteer/puppeteer#728

kimmobrunfeldt mentioned this issue Oct 13, 2017

Large HTML body causes Error: Navigation Timeout Exceeded: 30000ms exceeded alvarcarto/url-to-pdf-api#37

Closed

aslushnikov mentioned this issue Oct 23, 2017

refactor: migrate NavigatorWatcher to lifecycle events #1141

Merged

aslushnikov mentioned this issue Oct 24, 2017

feat(Page): teach Page.setContent to wait for resources to load #1152

Merged

aslushnikov closed this as completed in #1152 Oct 24, 2017

mariotsi mentioned this issue Nov 8, 2017

Revert "feat(Page): teach Page.setContent to wait for resources to lo… #1312

Merged

aslushnikov reopened this Nov 8, 2017

ebidel mentioned this issue Dec 21, 2017

setContent doesn't work since 0.13.0 #1638

Closed

aslushnikov mentioned this issue Jan 4, 2018

Frame's init event not emitted #1640

Closed

xyzdata mentioned this issue Oct 9, 2018

puppeteer & bugs xgqfrms/url-to-pdf#1

Open

aslushnikov mentioned this issue Oct 9, 2018

docs(examples): render custom HTML to PDF #3376

Closed

aslushnikov added a commit to aslushnikov/puppeteer that referenced this issue Nov 16, 2018

feat(chromium): roll Chromium to r608752

8179736

This roll includes: - https://crrev.com/608658 - DevTools: emit "init" lifecycle event when document gets opened References puppeteer#728

aslushnikov mentioned this issue Nov 16, 2018

feat(chromium): roll Chromium to r608752 #3555

Merged

aslushnikov added a commit that referenced this issue Nov 16, 2018

feat(chromium): roll Chromium to r608752 (#3555)

d9d9996

This roll includes: - https://crrev.com/608658 - DevTools: emit "init" lifecycle event when document gets opened References #728

aslushnikov mentioned this issue Nov 16, 2018

feat(page): support wait options for page.setContent #3557

Merged

aslushnikov added a commit to aslushnikov/puppeteer that referenced this issue Nov 20, 2018

feat(page): support waitUntil option for page.setContent

88ae816

Fixes puppeteer#728

aslushnikov closed this as completed in #3557 Nov 20, 2018

randytarampi mentioned this issue Dec 15, 2018

fix(export): Adjust createPdf for the puppeteer^1.11.0 API. jsonresume/resume-cli#294

Closed

deadwards90 mentioned this issue Jan 7, 2019

Use setContent for pdf and screenshot browserless/browserless#106

Merged

justisb mentioned this issue Feb 8, 2019

Directly set html content alvarcarto/url-to-pdf-api#95

Closed

Swaagie mentioned this issue Feb 18, 2019

[minor] provide default svg viewbox to fallback image godaddy/asset-system#19

Merged

guillaumev mentioned this issue Apr 10, 2019

OPS-5361 - wait for external resources to be loaded UN-OCHA/tools-snap-service#51

Merged

joone mentioned this issue Jun 27, 2019

White screenshot on second setContent #4480

Closed

leandro-costa mentioned this issue Nov 18, 2019

Sometimes plantuml diagram doesn't export to PDF qjebbs/vscode-markdown-extended#67

Closed

Norlandz mentioned this issue Mar 17, 2023

Ability to load HTML String as a page? #581

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Page.setContent should wait for resources to be loaded #728

Page.setContent should wait for resources to be loaded #728

aslushnikov commented Sep 9, 2017

aslushnikov commented Oct 4, 2017 •

edited by ebidel

cmazakas commented Oct 10, 2017 •

edited

murilozilli commented Nov 22, 2017

HanXHX commented Dec 7, 2017

Padam87 commented Dec 13, 2017

HanXHX commented Dec 15, 2017

aslushnikov commented Jan 10, 2018

tzieleniewski commented Oct 10, 2018

ObviouslyGreen commented Nov 16, 2018

cape-dev commented Nov 20, 2018 •

edited

cape-dev commented Nov 21, 2018 •

edited

aslushnikov commented Nov 21, 2018

tzieleniewski commented Nov 22, 2018

iksheth commented Aug 26, 2019

casesolved-co-uk commented May 16, 2024

Page.setContent should wait for resources to be loaded #728

Page.setContent should wait for resources to be loaded #728

Comments

aslushnikov commented Sep 9, 2017

aslushnikov commented Oct 4, 2017 • edited by ebidel

cmazakas commented Oct 10, 2017 • edited

murilozilli commented Nov 22, 2017

HanXHX commented Dec 7, 2017

Padam87 commented Dec 13, 2017

HanXHX commented Dec 15, 2017

aslushnikov commented Jan 10, 2018

tzieleniewski commented Oct 10, 2018

ObviouslyGreen commented Nov 16, 2018

cape-dev commented Nov 20, 2018 • edited

cape-dev commented Nov 21, 2018 • edited

aslushnikov commented Nov 21, 2018

tzieleniewski commented Nov 22, 2018

iksheth commented Aug 26, 2019

casesolved-co-uk commented May 16, 2024

aslushnikov commented Oct 4, 2017 •

edited by ebidel

cmazakas commented Oct 10, 2017 •

edited

cape-dev commented Nov 20, 2018 •

edited

cape-dev commented Nov 21, 2018 •

edited