By default the PDFreactor web service, which is used by the REST, JavaScript, Node.js, PHP, .NET, Python, Perl and Ruby APIs, has a maximum of 1024MB of memory assigned for its Java VM. For large documents, this might not be sufficient. To increase the amount of available memory, you can modify the "-Xmx1024m" parameter in the file "PDFreactor/jetty/start.d/main.ini".
For example, to increase the memory to 2GB, change the parameter to "-Xmx2048m" and restart the web service.
More information on increasing the memory for the PDFreactor Web Service can be found here.