Merge branch 'master' into lambda-provisioned-concurrency

docs/dashboard/cicd/best-practices.md

@@ -0,0 +1,21 @@
+<!--
+title: Serverless Dashboard - CI/CD Best Practices
+menuText: Best Practices
+layout: Doc
+-->
+
+<!-- DOCS-SITE-LINK:START automatically generated  -->
+
+### [Read this on the main serverless docs site](https://www.serverless.com/framework/docs/dashboard/cicd/best-practices/)
+
+<!-- DOCS-SITE-LINK:END -->
+
+# Serverless CI/CD Best Practices
+
+Serverless Framework Pro provides a lot of capabilities out of the box to help you manage and deploy
+your services. As your teams grow and the number of services grow, it can be difficult to know
+the best way to organize your services for scale.
+
+To help you manage and deploy your services at scale, check out the
+[Serverless CI/CD Workflow Guide](https://serverless.com/learn/guides/cicd/) for our recommendations
+on organizing your apps, servies, repos and automating your release process.

docs/providers/aws/cli-reference/deploy.md

@@ -64,3 +64,11 @@ serverless deploy --package /path/to/package/directory
 ```
 
 With this example, the packaging step will be skipped and the framework will start deploying the package from the `/path/to/package/directory` directory.
+
+### Environment variables
+
+- `SLS_AWS_MONITORING_FREQUENCY` allows the adjustment of the deployment monitoring frequency time in ms, default is `5000`.
+
+```bash
+SLS_AWS_MONITORING_FREQUENCY=10000 serverless deploy
+```

docs/providers/aws/events/apigateway.md

@@ -284,6 +284,8 @@ Please note that since you can't send multiple values for [Access-Control-Allow-
 
 Configuring the `cors` property sets [Access-Control-Allow-Origin](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Origin), [Access-Control-Allow-Headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers), [Access-Control-Allow-Methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Methods),[Access-Control-Allow-Credentials](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials) headers in the CORS preflight response.
 
+Please note that the [Access-Control-Allow-Credentials](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Credentials)-Header is omitted when not explicitly set to `true`.
+
 To enable the `Access-Control-Max-Age` preflight response header, set the `maxAge` property in the `cors` object:
 
 ```yml

docs/providers/aws/events/sqs.md

@@ -20,8 +20,6 @@ The ARN for the queue can be specified as a string, the reference to the ARN of
 
 **Note:** The `sqs` event will hook up your existing SQS Queue to a Lambda function. Serverless won't create a new queue for you.
 
-**IMPORTANT:** AWS is [not supporting FIFO queue](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html) to trigger Lambda function so your queue(s) **must be** a standard queue.
-
 ```yml
 functions:
   compute:

docs/providers/aws/events/streams.md

@@ -108,3 +108,23 @@ functions:
           arn: arn:aws:kinesis:region:XXXXXX:stream/foo
           batchWindow: 10
 ```
+
+## Setting the ParallelizationFactor
+
+The configuration below sets up a Kinesis stream event for the `preprocess` function which has a parallelization factor of 10 (default is 1).
+
+The `parallelizationFactor` property specifies the number of concurrent Lambda invocations for each shard of the Kinesis Stream.
+
+For more information, read the [AWS release announcement](https://aws.amazon.com/blogs/compute/new-aws-lambda-scaling-controls-for-kinesis-and-dynamodb-event-sources/) for this property.
+
+**Note:** The `stream` event will hook up your existing streams to a Lambda function. Serverless won't create a new stream for you.
+
+```yml
+functions:
+  preprocess:
+    handler: handler.preprocess
+    events:
+      - stream:
+          arn: arn:aws:kinesis:region:XXXXXX:stream/foo
+          parallelizationFactor: 10
+```

docs/providers/aws/events/websocket.md

@@ -212,3 +212,16 @@ provider:
 ```
 
 The log streams will be generated in a dedicated log group which follows the naming schema `/aws/websocket/{service}-{stage}`.
+
+The default log level will be INFO. You can change this to error with the following:
+
+```yml
+# serverless.yml
+provider:
+  name: aws
+  logs:
+    websocket:
+      level: ERROR
+```
+
+Valid values are INFO, ERROR.

docs/providers/aws/guide/serverless.yml.md

@@ -136,14 +136,15 @@ provider:
     apiGateway: true
     lambda: true # Optional, can be true (true equals 'Active'), 'Active' or 'PassThrough'
   logs:
-    restApi: # Optional configuration which specifies if API Gateway logs are used. This can either be set to true to use defaults, or configured via subproperties.
+    restApi: # Optional configuration which specifies if API Gateway logs are used. This can either be set to `true` to use defaults, or configured via subproperties.
       accessLogging: true # Optional configuration which enables or disables access logging. Defaults to true.
       format: 'requestId: $context.requestId' # Optional configuration which specifies the log format to use for access logging.
       executionLogging: true # Optional configuration which enables or disables execution logging. Defaults to true.
       level: INFO # Optional configuration which specifies the log level to use for execution logging. May be set to either INFO or ERROR.
       fullExecutionData: true # Optional configuration which specifies whether or not to log full requests/responses for execution logging. Defaults to true.
       role: arn:aws:iam::123456:role # Optional IAM role for ApiGateway to use when managing CloudWatch Logs
-    websocket: true # Optional configuration which specifies if Websockets logs are used
+    websocket: # Optional configuration which specifies if Websocket logs are used. This can either be set to `true` to use defaults, or configured via subproperties.
+      level: INFO # Optional configuration which specifies the log level to use for execution logging. May be set to either INFO or ERROR.
     frameworkLambda: true # Optional, whether to write CloudWatch logs for custom resource lambdas as added by the framework
 
 package: # Optional deployment packaging configuration

docs/providers/azure/guide/credentials.md

@@ -48,7 +48,7 @@ This will give you a code and prompt you to visit [aka.ms/devicelogin](https://a
 $ az account list
 {
   "cloudName": "AzureCloud",
-  "id": "c6e5c9a2-a4dd-4c05-81b4-6bed04f913ea",
+  "id": "<subscriptionId>",
   "isDefault": true,
   "name": "My Azure Subscription",
   "registeredProviders": [],

docs/providers/azure/guide/quick-start.md

@@ -156,10 +156,29 @@ The getting started walkthrough illustrates the interactive login experience, wh
    ```bash
    # Login to Azure
    $ az login
-   # Set Azure Subscription for which to create Service Principal
+   ```
+   This will yield something like:
+   ```json
+   [
+     {
+       "cloudName": "<cloudName>",
+       "id": "<subscription-id>",
+       "isDefault": true,
+       "name": "<name>",
+       "state": "<state>",
+       "tenantId": "<tenantId>",
+       "user": {
+         "name": "<name>",
+         "type": "<user>"
+       }
+     }
+   ]
+   ```
+3. Set Azure Subscription for which to create Service Principal
+   ```bash
    $ az account set -s <subscription-id>
    ```
-3. Generate Service Principal for Azure Subscription
+4. Generate Service Principal for Azure Subscription
    ```bash
    # Create SP with unique name
    $ az ad sp create-for-rbac --name <name>
@@ -174,7 +193,7 @@ The getting started walkthrough illustrates the interactive login experience, wh
      "tenant": "<tenantId>"
    }
    ```
-4. Set environment variables
+5. Set environment variables
 
    **Bash**
 

lib/classes/PluginManager.test.js

@@ -65,7 +65,7 @@ describe('PluginManager', () => {
     }
 
     functions() {
-      this.deployedFunctions = this.deployedFunctions + 1;
+      this.deployedFunctions += 1;
     }
   }
 
@@ -88,7 +88,7 @@ describe('PluginManager', () => {
     }
 
     functions() {
-      this.deployedFunctions = this.deployedFunctions + 1;
+      this.deployedFunctions += 1;
     }
   }
 
@@ -135,14 +135,14 @@ describe('PluginManager', () => {
 
     functions() {
       return new BbPromise(resolve => {
-        this.deployedFunctions = this.deployedFunctions + 1;
+        this.deployedFunctions += 1;
         return resolve();
       });
     }
 
     resources() {
       return new BbPromise(resolve => {
-        this.deployedResources = this.deployedResources + 1;
+        this.deployedResources += 1;
         return resolve();
       });
     }
@@ -190,11 +190,11 @@ describe('PluginManager', () => {
     }
 
     functions() {
-      this.deployedFunctions = this.deployedFunctions + 1;
+      this.deployedFunctions += 1;
     }
 
     resources() {
-      this.deployedResources = this.deployedResources + 1;
+      this.deployedResources += 1;
     }
   }
 
@@ -241,11 +241,11 @@ describe('PluginManager', () => {
     }
 
     functions() {
-      this.deployedFunctions = this.deployedFunctions + 1;
+      this.deployedFunctions += 1;
     }
 
     resources() {
-      this.deployedResources = this.deployedResources + 1;
+      this.deployedResources += 1;
     }
   }
 

lib/plugins/aws/customResources/generateZip.js

@@ -0,0 +1,47 @@
+'use strict';
+
+const os = require('os');
+const path = require('path');
+const { memoize } = require('lodash');
+const BbPromise = require('bluebird');
+const childProcess = BbPromise.promisifyAll(require('child_process'));
+const fse = BbPromise.promisifyAll(require('fs-extra'));
+const { version } = require('../../../../package');
+const getTmpDirPath = require('../../../utils/fs/getTmpDirPath');
+const createZipFile = require('../../../utils/fs/createZipFile');
+
+const srcDirPath = path.join(__dirname, 'resources');
+const cachedZipFilePath = path.join(
+  os.homedir(),
+  '.serverless/cache/custom-resources',
+  version,
+  'custom-resources.zip'
+);
+
+module.exports = memoize(() =>
+  fse
+    .lstatAsync(cachedZipFilePath)
+    .then(
+      stats => {
+        if (stats.isFile()) return true;
+        return false;
+      },
+      error => {
+        if (error.code === 'ENOENT') return false;
+        throw error;
+      }
+    )
+    .then(isCached => {
+      if (isCached) return cachedZipFilePath;
+      const ensureCachedDirDeferred = fse.ensureDirAsync(path.dirname(cachedZipFilePath));
+      const tmpDirPath = getTmpDirPath();
+      return fse
+        .copyAsync(srcDirPath, tmpDirPath)
+        .then(() => childProcess.execAsync('npm install', { cwd: tmpDirPath }))
+        .then(() => ensureCachedDirDeferred)
+        .then(() => createZipFile(tmpDirPath, cachedZipFilePath))
+        .then(() => cachedZipFilePath);
+    })
+);
+
+module.exports.cachedZipFilePath = cachedZipFilePath;

lib/plugins/aws/customResources/index.js

@@ -4,17 +4,7 @@ const path = require('path');
 const crypto = require('crypto');
 const BbPromise = require('bluebird');
 const fse = BbPromise.promisifyAll(require('fs-extra'));
-const childProcess = BbPromise.promisifyAll(require('child_process'));
-const getTmpDirPath = require('../../../utils/fs/getTmpDirPath');
-const createZipFile = require('../../../utils/fs/createZipFile');
-
-function copyCustomResources(srcDirPath, destDirPath) {
-  return fse.ensureDirAsync(destDirPath).then(() => fse.copyAsync(srcDirPath, destDirPath));
-}
-
-function installDependencies(dirPath) {
-  return childProcess.execAsync('npm install', { cwd: dirPath });
-}
+const generateZip = require('./generateZip');
 
 function addCustomResourceToService(awsProvider, resourceName, iamRoleStatements) {
   let functionName;
@@ -28,15 +18,12 @@ function addCustomResourceToService(awsProvider, resourceName, iamRoleStatements
   const shouldWriteLogs = providerConfig.logs && providerConfig.logs.frameworkLambda;
   const { Resources } = providerConfig.compiledCloudFormationTemplate;
   const customResourcesRoleLogicalId = awsProvider.naming.getCustomResourcesRoleLogicalId();
-  const srcDirPath = path.join(__dirname, 'resources');
-  const destDirPath = path.join(
+  const zipFilePath = path.join(
     serverless.config.servicePath,
     '.serverless',
-    awsProvider.naming.getCustomResourcesArtifactDirectoryName()
+    awsProvider.naming.getCustomResourcesArtifactName()
   );
-  const tmpDirPath = path.join(getTmpDirPath(), 'resources');
   const funcPrefix = `${serverless.service.service}-${cliOptions.stage}`;
-  const zipFilePath = `${destDirPath}.zip`;
   serverless.utils.writeFileDir(zipFilePath);
 
   // check which custom resource should be used
@@ -72,18 +59,17 @@ function addCustomResourceToService(awsProvider, resourceName, iamRoleStatements
 
   // TODO: check every once in a while if external packages are still necessary
   serverless.cli.log('Installing dependencies for custom CloudFormation resources...');
-  return copyCustomResources(srcDirPath, tmpDirPath)
-    .then(() => installDependencies(tmpDirPath))
-    .then(() => createZipFile(tmpDirPath, zipFilePath))
-    .then(outputFilePath => {
+  return generateZip().then(cachedZipFilePath => {
+    const zipFileBasename = path.basename(zipFilePath);
+    return fse.copyAsync(cachedZipFilePath, zipFilePath).then(() => {
       let S3Bucket = {
         Ref: awsProvider.naming.getDeploymentBucketLogicalId(),
       };
       if (serverless.service.package.deploymentBucket) {
         S3Bucket = serverless.service.package.deploymentBucket;
       }
       const s3Folder = serverless.service.package.artifactDirectoryName;
-      const s3FileName = outputFilePath.split(path.sep).pop();
+      const s3FileName = zipFileBasename;
       const S3Key = `${s3Folder}/${s3FileName}`;
 
       const cfnRoleArn = serverless.service.provider.cfnRole;
@@ -205,6 +191,7 @@ function addCustomResourceToService(awsProvider, resourceName, iamRoleStatements
         });
       }
     });
+  });
 }
 
 module.exports = {

lib/plugins/aws/customResources/index.test.js

@@ -2,7 +2,6 @@
 
 /* eslint-disable no-unused-expressions */
 
-const path = require('path');
 const fs = require('fs');
 const chai = require('chai');
 const sinon = require('sinon');
@@ -13,6 +12,7 @@ const CLI = require('../../../classes/CLI');
 const childProcess = BbPromise.promisifyAll(require('child_process'));
 const { createTmpDir } = require('../../../../tests/utils/fs');
 const { addCustomResourceToService } = require('./index.js');
+const customResourcesZipFilePath = require('./generateZip').cachedZipFilePath;
 
 const expect = chai.expect;
 chai.use(require('sinon-chai'));
@@ -103,13 +103,8 @@ describe('#addCustomResourceToService()', () => {
       ])
     ).to.be.fulfilled.then(() => {
       const { Resources } = serverless.service.provider.compiledCloudFormationTemplate;
-      const customResourcesZipFilePath = path.join(
-        tmpDirPath,
-        '.serverless',
-        'custom-resources.zip'
-      );
 
-      expect(execAsyncStub).to.have.callCount(3);
+      expect(execAsyncStub).to.have.callCount(1);
       expect(fs.existsSync(customResourcesZipFilePath)).to.equal(true);
       // S3 Lambda Function
       expect(Resources.CustomDashresourceDashexistingDashs3LambdaFunction).to.deep.equal({
@@ -270,13 +265,6 @@ describe('#addCustomResourceToService()', () => {
       ])
     ).to.be.fulfilled.then(() => {
       const { Resources } = serverless.service.provider.compiledCloudFormationTemplate;
-      const customResourcesZipFilePath = path.join(
-        tmpDirPath,
-        '.serverless',
-        'custom-resources.zip'
-      );
-
-      expect(execAsyncStub).to.have.callCount(3);
       expect(fs.existsSync(customResourcesZipFilePath)).to.equal(true);
       // S3 Lambda Function
       expect(Resources.CustomDashresourceDashexistingDashs3LambdaFunction).to.deep.equal({