Showing posts with label merge. Show all posts
Showing posts with label merge. Show all posts

Wednesday, March 21, 2012

Excluding duplicates after Multicast

I have to do various controls on a dataset - I created a multicast. After performing controls (one control per copy), I merge my (7) multicasted datasets using a Union All transformation. The problem I'm having are the duplicate rows created by merging the multicast copies.

How do I get rid of the duplicates? Is the Sort Transformation the solution by setting the option Remove rows with duplicate sort values to True? I have a unique key by which I'm able to discard the duplicates correctly. Are there any other ways (at a Union All level)? Is there sth like Union and Union All like in SQL?

I'm working on my 1st integration serv. project and it seems that more I work more questions I have. Shoudn't be the opposite? Thank you for the help.

See if the aggregate transformation can help you...

This paper has also some suggestions:

http://technet.microsoft.com/en-us/library/aa964137.aspx|||

Can you avoid creating duplicates in the first place? Perhpas the Conditional Split could be used instead of the Multicast?

I have found Sort to be the best de-duplication option, generally faster than the aggregate, but test it with your data if performance is an issue.

Excluding duplicates after Multicast

I have to do various controls on a dataset - I created a multicast. After performing controls (one control per copy), I merge my (7) multicasted datasets using a Union All transformation. The problem I'm having are the duplicate rows created by merging the multicast copies.

How do I get rid of the duplicates? Is the Sort Transformation the solution by setting the option Remove rows with duplicate sort values to True? I have a unique key by which I'm able to discard the duplicates correctly. Are there any other ways (at a Union All level)? Is there sth like Union and Union All like in SQL?

I'm working on my 1st integration serv. project and it seems that more I work more questions I have. Shoudn't be the opposite? Thank you for the help.

See if the aggregate transformation can help you...

This paper has also some suggestions:

http://technet.microsoft.com/en-us/library/aa964137.aspx|||

Can you avoid creating duplicates in the first place? Perhpas the Conditional Split could be used instead of the Multicast?

I have found Sort to be the best de-duplication option, generally faster than the aggregate, but test it with your data if performance is an issue.

Wednesday, March 7, 2012

Exception in EnumMergeDynamicSnapshotJobs

Hi,

I'm manually creating dynamic snapshots for a push merge subscription, however I often get an exception thrown when I call EnumMergeDynamicSnapshotJobs. The exception says:

Unable to cast object of type 'System.Byte' to type 'System.String'.

The stack trace is:

at System.Data.SqlClient.SqlBuffer.get_String()

at System.Data.SqlClient.SqlDataReader.GetString(Int32 i)

at Microsoft.SqlServer.Replication.ReplicationObject.SafeGetString(SqlDataReader lclReader, Int32 nColumn, Int32 nLength)

at Microsoft.SqlServer.Replication.MergePublication.EnumMergeDynamicSnapshotJobs()

at ART.AlphaRMS.DatabaseManagementLibrary.SMOCommands.CreateDynamicSnapshot(MergePublication publication, MergeSubscription subscription, String dynamicSnapshotLocation) in ...

Has anyone seen this exception here before?

Thanks for your help

Graham

Can you post how you are creating the dynamic snapshots.

Using the UI to create a partition and the job and then enumerating the job using EnumMergeDynamicSnapshotJobs() does not show me any problems. Please post your code snippet.

|||

Hi Mahesh, I create the snapshot in code (RMO). I do not see this error every time though, it is one of those intermitant problems!

public string CreateDynamicSnapshot(MergePublication publication, MergeSubscription subscription, string dynamicSnapshotLocation)

{

string snapshotLocation = string.Empty;

try

{

// Pre-generate the dynamic snapshot as it may be that we

// want to ship it off to a remote server.

if (!string.IsNullOrEmpty(dynamicSnapshotLocation))

{

string publicationName = publication.Name;

string publicationDbName = publication.DatabaseName;

string publisherName = publication.SqlServerName;

string distributorName = publication.SqlServerName;

// Set a weekly schedule for the filtered data snapshot.

ReplicationAgentSchedule schedule = new ReplicationAgentSchedule();

schedule.FrequencyType = ScheduleFrequencyType.Weekly;

schedule.FrequencyRecurrenceFactor = 1;

schedule.FrequencyInterval = Convert.ToInt32(0x001);

// Set the value of Hostname that defines the data partition.

MergePartition partition = new MergePartition();

partition.DynamicFilterHostName = subscription.HostName;

MergeDynamicSnapshotJob snapshotAgentJob = new MergeDynamicSnapshotJob();

snapshotAgentJob.DynamicFilterHostName = subscription.HostName;

// Before creating the new job, make sure any old instances are deleted.

foreach (MergeDynamicSnapshotJob agentJob in publication.EnumMergeDynamicSnapshotJobs())

{

if (string.Compare(agentJob.DynamicFilterHostName, snapshotAgentJob.DynamicFilterHostName, true) == 0)

{

logger.Debug("Found existing dynamic snapshot job, deleting it.");

publication.RemoveMergeDynamicSnapshotJob(agentJob.Name);

}

}

logger.Debug("Creating new dynamic snapshot job for partition " + partition.DynamicFilterHostName);

// Create the partition for the publication with the defined schedule.

publication.AddMergePartition(partition);

publication.AddMergeDynamicSnapshotJob(snapshotAgentJob, schedule);

snapshotLocation = StartDynamicSnapshotJob(publication, subscription, snapshotLocation);

}

}

catch (Exception ex)

{

logger.Error("Failed to create the new dynamic snapshot.", ex);

throw;

}

return snapshotLocation;

}

Sunday, February 26, 2012

Exception error in snapshot agent

Greetings,
We are using SQLServer 2000 and
we are replicating one server to another using merge replication... It has
been working fine
but all of the sudden, we are getting "An exception occurred in the Snapshot
subsystem"...
I've tried re-creating the publication a number of times but always the same
error with the Snapshot...
I've looked through all the log files but can't find any additional info
about the error.
Does anyone have any suggestion as to where or what to look for?
Thanks in advance.
Hi Steve,
From what you described below, it looks like the replication subsystem for
launching the snapshot agent from SQLServerAgent is crashing. You may want to
try stopping and restarting the SQLServerAgent service and see if that
resolves the problem.
If you are running a version of SQL2000 < SP4, the crash may also be due to
an inproperly handled COM error in the replication subsystem of
SQLServerAgent. As such, upgrading your SQL2000 instance to SP4 may provide
you with a more descriptive error of the underlying problem. If you do get a
more descriptive error after upgrading to SP4, you may be able to find a
solution by searching for the error string on the Microsoft Support web site.
HTH
-Raymond
"Steve Stoenner" wrote:

> Greetings,
> We are using SQLServer 2000 and
> we are replicating one server to another using merge replication... It has
> been working fine
> but all of the sudden, we are getting "An exception occurred in the Snapshot
> subsystem"...
> I've tried re-creating the publication a number of times but always the same
> error with the Snapshot...
> I've looked through all the log files but can't find any additional info
> about the error.
> Does anyone have any suggestion as to where or what to look for?
> Thanks in advance.
>
>

Friday, February 24, 2012

Excel XSLT?

What's the best way to combine XML INTO an excel document? I'm using the SSIS XML Task to do this.

There are a few options, XSLT, merge etc. Which one will take raw xml (from a database query for instance) and, combined with, an XSL document?, create a spreadsheet in excel?

Please help,

Phil

I'll answer my own question again. The operant type should be XSLT and with the correct XSL you can create an excel document using raw XML from SQL.

Phil

Excel XSLT?

What's the best way to combine XML INTO an excel document? I'm using the SSIS XML Task to do this.

There are a few options, XSLT, merge etc. Which one will take raw xml (from a database query for instance) and, combined with, an XSL document?, create a spreadsheet in excel?

Please help,

Phil

I'll answer my own question again. The operant type should be XSLT and with the correct XSL you can create an excel document using raw XML from SQL.

Phil