OutOffMemory Exception

Hi all,
I have an exception as follows:
java.lang.OutOfMemoryError
at com.microsoft.util.UtilPagedTempBuffer.compressBlo ckList(Unknown Source)
at com.microsoft.util.UtilPagedTempBuffer.getBlock(Un known Source)
at com.microsoft.util.UtilPagedTempBuffer.write(Unkno wn Source)
at com.microsoft.util.UtilPagedTempBuffer.write(Unkno wn Source)
at com.microsoft.util.UtilByteArrayDataProvider.recei ve(Unknown Source)
at com.microsoft.util.UtilByteOrderedDataReader.recei ve(Unknown Source)
at com.microsoft.jdbc.sqlserver.tds.TDSRPCRequest.sub mitRequest(Unknown
Source)
at com.microsoft.jdbc.sqlserver.tds.TDSCursorRequest. openCursor(Unknown
Source)
at com.microsoft.jdbc.sqlserver.SQLServerImplStatemen t.execute(Unknown
Source)
at com.microsoft.jdbc.base.BaseStatement.commonExecut e(Unknown Source)
at com.microsoft.jdbc.base.BaseStatement.executeQuery Internal(Unknown
Source)
at com.microsoft.jdbc.base.BasePreparedStatement.exec uteQuery(Unknown
Source)
De query used is a simple select statement:
select t1.pm_key, t2.type
from table1 t1, table2 t2
where t1.pm_key = t2.pm_key
and t2.status <= '160-00'
and t1.from_key = 'P000003456'
My questions:
1. Is it a driver problem?
2. What is the driver doing at that point?
3. How can I avoid this ? (i.e. re-write the query)
4. Can I give the process more memory (other than -Xmx<amount>)?
Any tips/hints are very welcome.
TIA.
Frank.
Hi.
How much data is being returned?
I would suggest adding the connection property
selectMethod=cursor to your connection properties.
That might save client-side memory.
Joe Weinstein at BEA
Frank Brouwer wrote:

> Hi all,
> I have an exception as follows:
> java.lang.OutOfMemoryError
> at com.microsoft.util.UtilPagedTempBuffer.compressBlo ckList(Unknown Source)
> at com.microsoft.util.UtilPagedTempBuffer.getBlock(Un known Source)
> at com.microsoft.util.UtilPagedTempBuffer.write(Unkno wn Source)
> at com.microsoft.util.UtilPagedTempBuffer.write(Unkno wn Source)
> at com.microsoft.util.UtilByteArrayDataProvider.recei ve(Unknown Source)
> at com.microsoft.util.UtilByteOrderedDataReader.recei ve(Unknown Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSRPCRequest.sub mitRequest(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSCursorRequest. openCursor(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.SQLServerImplStatemen t.execute(Unknown
> Source)
> at com.microsoft.jdbc.base.BaseStatement.commonExecut e(Unknown Source)
> at com.microsoft.jdbc.base.BaseStatement.executeQuery Internal(Unknown
> Source)
> at com.microsoft.jdbc.base.BasePreparedStatement.exec uteQuery(Unknown
> Source)
> De query used is a simple select statement:
> select t1.pm_key, t2.type
> from table1 t1, table2 t2
> where t1.pm_key = t2.pm_key
> and t2.status <= '160-00'
> and t1.from_key = 'P000003456'
> My questions:
> 1. Is it a driver problem?
> 2. What is the driver doing at that point?
> 3. How can I avoid this ? (i.e. re-write the query)
> 4. Can I give the process more memory (other than -Xmx<amount>)?
>
> Any tips/hints are very welcome.
> TIA.
> Frank.
>
|||Hi Joe,
Thanks for the tip, but I already use that selection method on the
connection. The query is supposed to retrieve about 1,500 reccords and the
tables contain about 30,000 reccords. It also takes a long time of usage
of the system before the message appears so I also checked for memory leaks
(closing connections and prepared statements etc..).
If I look at the log file I have, there is an other message just before the
OutOffMemory message:
java.sql.SQLException: [Microsoft][SQLServer 2000 Driver for
JDBC][SQLServer]Transaction (Process ID 69) was deadlocked on {lock}
resources with another process and has been chosen as the deadlock victim.
Rerun the transaction.
at com.microsoft.jdbc.base.BaseExceptions.createExcep tion(Unknown Source)
at com.microsoft.jdbc.base.BaseExceptions.getExceptio n(Unknown Source)
at com.microsoft.jdbc.sqlserver.tds.TDSRequest.proces sErrorToken(Unknown
Source)
at com.microsoft.jdbc.sqlserver.tds.TDSRequest.proces sReplyToken(Unknown
Source)
at com.microsoft.jdbc.sqlserver.tds.TDSRPCRequest.pro cessReplyToken(Unknown
Source)
at com.microsoft.jdbc.sqlserver.tds.TDSRequest.proces sReply(Unknown Source)
at com.microsoft.jdbc.sqlserver.tds.TDSCursorRequest. openCursor(Unknown
Source)
at com.microsoft.jdbc.sqlserver.SQLServerImplStatemen t.execute(Unknown
Source)
at com.microsoft.jdbc.base.BaseStatement.commonExecut e(Unknown Source)
at com.microsoft.jdbc.base.BaseStatement.executeQuery Internal(Unknown
Source)
at com.microsoft.jdbc.base.BasePreparedStatement.exec uteQuery(Unknown
Source)
Do you think that it can cause this problem?
If I look at:
at com.microsoft.util.UtilPagedTempBuffer.compressBlo ckList(Unknown Source)
I think that the JDBC driver is using some kind of internal (Hash?) table to
do some sorting, is it possible that this has some kind of limmit?
Thanks for your help.
Frank.
"Joe Weinstein" <joeNOSPAM@.bea.com> wrote in message
news:408FDABC.5060508@.bea.com...[vbcol=seagreen]
> Hi.
> How much data is being returned?
> I would suggest adding the connection property
> selectMethod=cursor to your connection properties.
> That might save client-side memory.
> Joe Weinstein at BEA
> Frank Brouwer wrote:
Source)
>
|||Frank Brouwer wrote:

> Hi Joe,
> Thanks for the tip, but I already use that selection method on the
> connection. The query is supposed to retrieve about 1,500 reccords and the
> tables contain about 30,000 reccords. It also takes a long time of usage
> of the system before the message appears so I also checked for memory leaks
> (closing connections and prepared statements etc..).
> If I look at the log file I have, there is an other message just before the
> OutOffMemory message:
> java.sql.SQLException: [Microsoft][SQLServer 2000 Driver for
> JDBC][SQLServer]Transaction (Process ID 69) was deadlocked on {lock}
> resources with another process and has been chosen as the deadlock victim.
> Rerun the transaction.
> at com.microsoft.jdbc.base.BaseExceptions.createExcep tion(Unknown Source)
> at com.microsoft.jdbc.base.BaseExceptions.getExceptio n(Unknown Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSRequest.proces sErrorToken(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSRequest.proces sReplyToken(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSRPCRequest.pro cessReplyToken(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSRequest.proces sReply(Unknown Source)
> at com.microsoft.jdbc.sqlserver.tds.TDSCursorRequest. openCursor(Unknown
> Source)
> at com.microsoft.jdbc.sqlserver.SQLServerImplStatemen t.execute(Unknown
> Source)
> at com.microsoft.jdbc.base.BaseStatement.commonExecut e(Unknown Source)
> at com.microsoft.jdbc.base.BaseStatement.executeQuery Internal(Unknown
> Source)
> at com.microsoft.jdbc.base.BasePreparedStatement.exec uteQuery(Unknown
> Source)
> Do you think that it can cause this problem?
>
no.

> If I look at:
> at com.microsoft.util.UtilPagedTempBuffer.compressBlo ckList(Unknown Source)
> I think that the JDBC driver is using some kind of internal (Hash?) table to
> do some sorting, is it possible that this has some kind of limmit?
No limit as such, but maybe inefficient code for the size of the task.
If you can make a standalone simple jdbc program that creates
the table, loops populating it, and then does the call that causes the mem
problem, MS would take it from there...
Joe

> Thanks for your help.
> Frank.
> "Joe Weinstein" <joeNOSPAM@.bea.com> wrote in message
> news:408FDABC.5060508@.bea.com...
>
> Source)
>
>

Out of memory Exception when Running a package with XML Task

Hi..

I'm running a package that has a XML Task in the control flow. This task tranforms a XML file with a XSLT.

The file is about 2 megs on a daily basis, but at the end of the month there is a full dump of data that makes the file to be around 400 megs. There is where my problem is.

I run this on my 2 GB memory workstation and when the memory gauge on the task manager reaches about 1.5gb the package fails with an "Out of memory exception".

I also run this package on a 8GB Ram server, and same applies.

Is there any way of making this package utilize all the available memory, I even increased the virtual memory to see if that helped my issue, but nothing.

Thanks

Are you running on a 32-bit or 64-bit OS? 32-bit servers are generally limited to about 2 gb of addressable memory, unless you have made some changes to the server configuration.|||

I'm running 32-bit... I wasn't aware of that!! I hope my admins knew that and made whatever change (I'm been highly opomistic, givin them the benefit of the doubt).

**UPDATE**

Yes... The server has the "/PAE" switch in the boot.ini, so its configured to use the 8 gigs of memory available to it.

|||Are other things running on the server - for example, the SQL Server relational engine? How much memory is allocated to it?|||

I'm looking at the Performance Tab in Task Manager and there is

Total Physical Memory: 8.32 Gigs

Available Physical Mem: 7.12 Gigs

From that 1 gig of memory taken is about 600 Megs devoted to System Cache and the rest is commited to running programs like SQL Server.

The only SQL server 2005 services that are running are

SSIS (of course)

Full Text Search

SQL Server

SQL Server Agent

Again... Thanks for all your help

|||

This is a good work around.

I used a Script Task (I find myself using it more and more as I find limitations in SSIS) and with 3 simple lines of code I resolved the issue.

Code Snippet

Dim xslt As New XslTransform()
xslt.Load("C:\Temp\Template.xslt")
xslt.Transform("C:\Temp\XMLSource.xml", "C:\Temp\XMLSource_Revised.xml")

This code performs the task in 5 mins -/+ consumming about 900 megs of memory space, for a 500 meg file.

Out of memory Exception when Running a package with XML Task

Hi..

I'm running a package that has a XML Task in the control flow. This task tranforms a XML file with a XSLT.

The file is about 2 megs on a daily basis, but at the end of the month there is a full dump of data that makes the file to be around 400 megs. There is where my problem is.

I run this on my 2 GB memory workstation and when the memory gauge on the task manager reaches about 1.5gb the package fails with an "Out of memory exception".

I also run this package on a 8GB Ram server, and same applies.

Is there any way of making this package utilize all the available memory, I even increased the virtual memory to see if that helped my issue, but nothing.

Thanks

Are you running on a 32-bit or 64-bit OS? 32-bit servers are generally limited to about 2 gb of addressable memory, unless you have made some changes to the server configuration.|||

I'm running 32-bit... I wasn't aware of that!! I hope my admins knew that and made whatever change (I'm been highly opomistic, givin them the benefit of the doubt).

**UPDATE**

Yes... The server has the "/PAE" switch in the boot.ini, so its configured to use the 8 gigs of memory available to it.

|||Are other things running on the server - for example, the SQL Server relational engine? How much memory is allocated to it?|||

I'm looking at the Performance Tab in Task Manager and there is

Total Physical Memory: 8.32 Gigs

Available Physical Mem: 7.12 Gigs

From that 1 gig of memory taken is about 600 Megs devoted to System Cache and the rest is commited to running programs like SQL Server.

The only SQL server 2005 services that are running are

SSIS (of course)

Full Text Search

SQL Server

SQL Server Agent

Again... Thanks for all your help

|||

This is a good work around.

I used a Script Task (I find myself using it more and more as I find limitations in SSIS) and with 3 simple lines of code I resolved the issue.

Code Snippet

Dim xslt As New XslTransform()
xslt.Load("C:\Temp\Template.xslt")
xslt.Transform("C:\Temp\XMLSource.xml", "C:\Temp\XMLSource_Revised.xml")

This code performs the task in 5 mins -/+ consumming about 900 megs of memory space, for a 500 meg file.

Out of Memory Exception

I'm getting an out of memory exception when designing by report model and
selecting save. I'm using 64bit windows xp 4Gb Ram and two dual core xeons
so presumably this is a bug! Anyone have any ideas?Hello Phil,
I would like to know whether there is any related error in the Report Log
File.
Would you please try to check the Report log File and post the error here
for me to troubleshooting?
By default, the log file is located at \Microsoft SQL Server\<SQL Server
Instance>\Reporting Services\LogFiles.
Please let me know the resault. Thank you!
Sincerely,
Wei Lu
Microsoft Online Community Support
==================================================
Get notification to my posts through email? Please refer to
http://msdn.microsoft.com/subscriptions/managednewsgroups/default.aspx#notif
ications.
Note: The MSDN Managed Newsgroup support offering is for non-urgent issues
where an initial response from the community or a Microsoft Support
Engineer within 1 business day is acceptable. Please note that each follow
up response may take approximately 2 business days as the support
professional working with you may need further investigation to reach the
most efficient resolution. The offering is not appropriate for situations
that require urgent, real-time or phone-based interactions or complex
project analysis and dump analysis issues. Issues of this nature are best
handled working with a dedicated Microsoft Support Engineer by contacting
Microsoft Customer Support Services (CSS) at
http://msdn.microsoft.com/subscriptions/support/default.aspx.
==================================================(This posting is provided "AS IS", with no warranties, and confers no
rights.)|||Hi Phil,
How is everything going? Please feel free to let me know if you need any
assistance.
Sincerely,
Wei Lu
Microsoft Online Community Supportsql