Hey guys,
Good afternoon!

In this post I will talk about a highly requested feature for Microsoft, which it has not yet created natively in SQL Server, which is data concatenation using groupings, already present in other DBMSs such as MySQL (GROUP_CONCAT), Oracle (XMLAGG) and PostgreeSQL (STRING_AGG or ARRAY_TO_STRING(ARRAY_AGG())).

Many people think that this feature is the CONCAT() function, introduced in SQL Server 2012, but it only allows the concatenation of multiple columns of a row into one column and not the concatenation of multiple rows into one column.

Concatenating columns into grouped strings consists of transforming lines into a concatenated string, so that you can group the values ​​by some field, and the other column is concatenated, as shown below:

SQL Server - Grouped Concatenation convert rows into string
SQL Server - Grouped Concatenation convert rows into string

To create our test table, use the commands below:

IF (OBJECT_ID('dbo.Teste_Group_Concat') IS NOT NULL) DROP TABLE dbo.Teste_Group_Concat
CREATE TABLE dbo.Teste_Group_Concat(
    Categoria VARCHAR(100),
    Descricao VARCHAR(100)
)

INSERT INTO dbo.Teste_Group_Concat( Categoria, Descricao )
VALUES ('Brinquedo', 'Bola'),
('Brinquedo', 'Carrinho'),
('Brinquedo', 'Boneco'),
('Brinquedo', 'Jogo'),
('Cama e Mesa', 'Toalha'),
('Cama e Mesa', 'Edredom'),
('Informatica', 'Teclado'),
('Informatica', 'Mouse'),
('Informatica', 'HD'),
('Informatica', 'CPU'),
('Informatica', 'Memoria'),
('Informatica', 'Placa-Mae'),
(NULL, 'TV')
COALESCE

How to concatenate columns into grouped strings using COALESCE

DECLARE @Nomes VARCHAR(MAX)

SELECT 
    @Nomes = COALESCE(@Nomes + ', ', '') + Descricao
FROM 
    dbo.Teste_XML

SELECT @Nomes

SQL Server - Grouped Concatenation convert rows into string - COALESCE
SQL Server - Grouped Concatenation convert rows into string - COALESCE

As we can see above, using the COALESCE function, we always need to store the results in a variable, and for this reason, we are unable to obtain the results grouped by category, as we need. If you just want to convert columns into a string, without grouping, this solution will suit you, but not what we need for this post.

STUFF + FOR XML PATH

How to concatenate columns into grouped strings using STUFF + FOR XML PATH

A practical and performative way to solve this problem is to use FOR XML PATH and STUFF to perform grouped concatenation, so that the result is exactly what we expected:

SELECT  
    Categoria,
    STUFF((
        SELECT ', ' + B.Descricao 
        FROM dbo.Teste_Group_Concat B 
        WHERE ISNULL(B.Categoria, '') = ISNULL(A.Categoria, '')
        ORDER BY B.Descricao 
        FOR XML PATH('')), 1, 2, ''
    ) AS Descricao
FROM
    dbo.Teste_Group_Concat A
GROUP BY 
    Categoria
ORDER BY 
    Categoria

SQL Server - Grouped Concatenation convert rows into string - STUFF FOR XML PATH
SQL Server - Grouped Concatenation convert rows into string - STUFF FOR XML PATH

UPDATE

How to concatenate columns into grouped strings using UPDATE

DECLARE 
    @Categoria VARCHAR(MAX),
    @Descricoes VARCHAR(MAX)
 
DECLARE @Tabela TABLE (
    Categoria VARCHAR(200),
    Descricao VARCHAR(200),
    Descricoes VARCHAR(200)
)
 
INSERT @Tabela ( 
    Categoria,
    Descricao
)
SELECT  
    Categoria,
    Descricao
FROM
    dbo.Teste_Group_Concat
ORDER BY 
    Categoria,
    Descricao
 

UPDATE 
    @Tabela
SET
    @Descricoes = Descricoes = COALESCE(CASE COALESCE(@Categoria, '')
                                WHEN Categoria THEN @Descricoes + ', ' + Descricao
                                ELSE Descricao
                            END, ''),
    @Categoria = ISNULL(Categoria, '')


SELECT  
    Categoria,
    Descricoes = MAX(Descricoes)
FROM
    @Tabela
GROUP BY 
    Categoria
ORDER BY 
    Categoria

SQL Server - Grouped Concatenation convert rows into string - UPDATE
SQL Server - Grouped Concatenation convert rows into string - UPDATE

Another solution that allowed us to generate data as our needs demand.

LOOPING WITH CURSOR

How to concatenate columns into grouped strings using LOOPING with CURSOR

DECLARE @Tabela TABLE (
    Categoria VARCHAR(MAX),
    Descricoes VARCHAR(MAX)
)
 
INSERT @Tabela ( 
    Categoria,
    Descricoes
)
SELECT  
    Categoria,
    ''
FROM
    dbo.Teste_Group_Concat
GROUP BY 
    Categoria
 

DECLARE 
    @Categoria VARCHAR(MAX),
    @Descricao VARCHAR(MAX),
    @Descricoes VARCHAR(MAX)
 
DECLARE c CURSOR LOCAL FAST_FORWARD
FOR

    SELECT  
        Categoria,
        Descricao
    FROM
        dbo.Teste_Group_Concat
    ORDER BY 
        Categoria,
        Descricao
 

OPEN c
 
FETCH c INTO @Categoria, @Descricao
 
WHILE @@FETCH_STATUS = 0
BEGIN

    UPDATE  @Tabela
    SET     Descricoes += ', ' + @Descricao
    WHERE   ISNULL(Categoria, '') = ISNULL(@Categoria, '')
 
    FETCH c INTO @Categoria, @Descricao
END
 
CLOSE c
DEALLOCATE c

SELECT  
    Categoria,
    Descricoes = STUFF(Descricoes, 1, 1, '')
FROM
    @Tabela
ORDER BY 
    Categoria

SQL Server - Grouped Concatenation convert rows into string - CURSOR
SQL Server - Grouped Concatenation convert rows into string - CURSOR

This solution also met what we needed, but like all looping, it is not performant. I personally abhor creating cursors (unless you are STILL on SQL Server 2000), in the latter case, I prefer to use WHILE, but we have better solutions here in this post.

LOOPING WITH WHILE

How to concatenate columns into grouped strings using LOOPING with WHILE

DECLARE @Tabela_Original TABLE (
    Id INT IDENTITY(1, 1),
    Categoria VARCHAR(MAX),
    Descricao VARCHAR(MAX)
)
INSERT @Tabela_Original ( 
    Categoria,
    Descricao
)
SELECT  
    Categoria,
    Descricao
FROM
    dbo.Teste_Group_Concat
ORDER BY
    Categoria


DECLARE @Tabela TABLE (
    Id INT IDENTITY(1, 1),
    Categoria VARCHAR(MAX),
    Descricoes VARCHAR(MAX)
)

INSERT @Tabela ( 
    Categoria,
    Descricoes
)
SELECT  
    Categoria,
    ''
FROM
    dbo.Teste_Group_Concat
GROUP BY 
    Categoria
 
DECLARE 
    @Categoria VARCHAR(MAX),
    @Descricao VARCHAR(MAX),
    @Descricoes VARCHAR(MAX),
    @Contador INT = 1,
    @Numero_Linhas INT = (SELECT COUNT(*) FROM @Tabela_Original)
 
 WHILE(@Contador <= @Numero_Linhas)
 BEGIN

    SELECT  
        @Categoria = ISNULL(Categoria, ''),
        @Descricao = Descricao
    FROM
        @Tabela_Original
    WHERE
        Id = @Contador
    ORDER BY 
        Categoria,
        Descricao
    

    UPDATE  @Tabela
    SET     Descricoes += ', ' + @Descricao
    WHERE   ISNULL(Categoria, '') = ISNULL(@Categoria, '')


    SET @Contador = @Contador + 1

END


SELECT  
    Categoria,
    Descricoes = STUFF(Descricoes, 1, 1, '')
FROM
    @Tabela
ORDER BY 
    Categoria

SQL Server - Grouped Concatenation convert rows into string - WHILE
SQL Server - Grouped Concatenation convert rows into string - WHILE

Solution met, but like all looping, it is not performant.

RECURSIVE CTE

How to concatenate columns into grouped strings using Recursive CTE

;WITH Dados as 
(
    SELECT 
        Categoria, 
        CONVERT(NVARCHAR(MAX), Descricao) AS Descricao,
        ROW_NUMBER() OVER (PARTITION BY Categoria ORDER BY Descricao) AS Linha
    FROM 
        dbo.Teste_Group_Concat
),
Tabela1 AS 
(
    SELECT Categoria, Descricao, Linha FROM Dados WHERE Linha = 1
),
Resultado AS
(
    SELECT Categoria, Descricao, Linha FROM Tabela1 WHERE Linha = 1
    
    UNION ALL

    SELECT Dados.Categoria, Resultado.Descricao + N', ' + Dados.Descricao, Dados.Linha
    FROM Dados 
    JOIN Resultado ON Resultado.Categoria = Dados.Categoria AND Dados.Linha = Resultado.Linha + 1
)
SELECT 
    Categoria, 
    Descricoes = MAX(Descricao)
FROM 
    Resultado
GROUP BY 
    Categoria 
ORDER BY 
    Categoria
OPTION (MAXRECURSION 0);

SQL Server - Grouped Concatenation convert rows into string - RECURSIVE CTE
SQL Server - Grouped Concatenation convert rows into string - RECURSIVE CTE

This solution uses the recursion technique to solve our problem without needing looping. Although it is not very easy to understand at first, it is a practical and very interesting solution.

SCALAR FUNCTION (UDF)

How to concatenate columns into grouped strings using Scalar Function (UDF)

CREATE FUNCTION dbo.fncConcatCategoria ( 
    @Ds_Categoria VARCHAR(MAX) 
)
RETURNS VARCHAR(MAX)
WITH SCHEMABINDING
AS
BEGIN

    DECLARE @Retorno VARCHAR(MAX);
 
    SELECT  
        @Retorno = COALESCE(@Retorno + ', ', '') + Descricao
    FROM
        dbo.Teste_Group_Concat
    WHERE
        ISNULL(Categoria, '') = ISNULL(@Ds_Categoria, '')
    ORDER BY 
        Descricao
 
    RETURN @Retorno

END
GO

SELECT 
    Categoria,
    dbo.fncConcatCategoria(Categoria) AS Descricao
FROM 
    dbo.Teste_Group_Concat
GROUP BY
    Categoria

SQL Server - Grouped Concatenation convert rows into string - SCALAR FUNCTION UDF
SQL Server - Grouped Concatenation convert rows into string - SCALAR FUNCTION UDF

Very useful function and once created, it becomes very practical to use. The problem with this solution is that it is not very performant and requires you to create a function for each table that you need to carry out this solution, in other words, it ends up not being very generic and you would need to create several functions that do practically the same thing in your database.

SQL CLR (C#)

How to concatenate columns into grouped strings using SCL CLR (C#)

using System;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Text;

[Serializable]
[SqlUserDefinedAggregate(
    Format.UserDefined,
    IsInvariantToNulls = false,
    IsInvariantToDuplicates = false,
    IsInvariantToOrder = true,
    MaxByteSize = -1)]
public struct concat : IBinarySerialize, INullable
{
    private StringBuilder _accumulator;
    private string _delimiter;

    public bool IsNull { get; private set; }

    public void Init()
    {
        _accumulator = new StringBuilder();
        _delimiter = string.Empty;
        this.IsNull = true;
    }

    public void Accumulate([SqlFacet(MaxSize = -1)]SqlString Value, [SqlFacet(MaxSize = -1)]SqlString Delimiter)
    {

        if (Value.IsNull) return;

        if (!Delimiter.IsNull & Delimiter.Value.Length > 0)
        {
            _delimiter = Delimiter.Value;
            if (_accumulator.Length > 0)
                _accumulator.Append(Delimiter.Value);

        }

        _accumulator.Append(Value.Value);
        IsNull = false;
    }
    
    public void Merge(concat Group)
    {
        if (_accumulator.Length > 0
            & Group._accumulator.Length > 0) _accumulator.Append(_delimiter);

        _accumulator.Append(Group._accumulator.ToString());

    }

    public SqlString Terminate()
    {
        return new SqlString(_accumulator.ToString());
    }

    void IBinarySerialize.Read(System.IO.BinaryReader r)
    {
        _delimiter = r.ReadString();
        _accumulator = new StringBuilder(r.ReadString());

        if (_accumulator.Length != 0) this.IsNull = false;
    }

    void IBinarySerialize.Write(System.IO.BinaryWriter w)
    {
        w.Write(_delimiter);
        w.Write(_accumulator.ToString());
    }
}

SQL Server - Grouped Concatenation convert rows into string - SQL CLR CSHARP
SQL Server - Grouped Concatenation convert rows into string - SQL CLR CSHARP

My favorite solution. Practical, extremely fast, generic and solves our problem. Don't know what CLR is or do you know, but don't know how to implement it in your database? Find out more by visiting my post Introduction to SQL CLR (Common Language Runtime) in SQL Server.

Performance Considerations

Let's now measure our solutions and find out which are the fastest and which are the slowest.

CLR

SQL Server - Grouped Concatenation convert rows into string - Performance - CLR
SQL Server - Grouped Concatenation convert rows into string - Performance - CLR

STUFF + FOR XML PATH

SQL Server - Grouped Concatenation convert rows into string - Performance - STUFF FOR XML PATH
SQL Server - Grouped Concatenation convert rows into string - Performance - STUFF FOR XML PATH

UPDATE

SQL Server - Grouped Concatenation convert rows into string - Performance - UPDATE
SQL Server - Grouped Concatenation convert rows into string - Performance - UPDATE

Recursive CTE

SQL Server - Grouped Concatenation convert rows into string - Performance - CTE
SQL Server - Grouped Concatenation convert rows into string - Performance - CTE

UDF scalar function

SQL Server - Grouped Concatenation convert rows into string - Performance - UDF Function
SQL Server - Grouped Concatenation convert rows into string - Performance - UDF Function

Loop While

SQL Server - Grouped Concatenation convert rows into string - Performance - WHILE
SQL Server - Grouped Concatenation convert rows into string - Performance - WHILE

Loop with Cursor

SQL Server - Grouped Concatenation convert rows into string - Performance - CURSOR
SQL Server - Grouped Concatenation convert rows into string - Performance - CURSOR

Right away we can eliminate loopings, which were much worse. What happens if we insert more records? Like 850,000 lines... How will the performance be?

CLR

SQL Server - Grouped Concatenation convert rows into string - Performance - CLR2
SQL Server - Grouped Concatenation convert rows into string - Performance - CLR2

STUFF + FOR XML PATH

SQL Server - Grouped Concatenation convert rows into string - Performance - STUFF FOR XML PATH
SQL Server - Grouped Concatenation convert rows into string - Performance - STUFF FOR XML PATH

As no other method was able to finish processing in less than 1 minute, despite having used 850,000 lines with the CLR and FOR XML PATH, I will only use 158,000 lines (27%) for the rest of the methods to try to analyze the results. Let's see.

UPDATE

SQL Server - Grouped Concatenation convert rows into string - Performance - UPDATE
SQL Server - Grouped Concatenation convert rows into string - Performance - UPDATE

The UPDATE time wasn't that bad... 5.7s for 158,000 lines is a reasonable time.

UDF scalar function

SQL Server - Grouped Concatenation convert rows into string - Performance - UDF Function3
SQL Server - Grouped Concatenation convert rows into string - Performance - UDF Function3

Recursive CTE

SQL Server - Grouped Concatenation convert rows into string - Performance - CTE Recursive
SQL Server - Grouped Concatenation convert rows into string - Performance - CTE Recursive

Well, I tried waiting for the Recursive CTE... I even went to lunch and left it running here, but after 2 hours and 22 minutes I couldn't wait any longer to finish processing the 158 thousand lines (remembering that the CLR processed 850,000 in 3s and the FOR XML in 1.5s)..

Apparently, our winner was FOR XML PATH, but if we analyze the results, the CLR delivered the correct result, even with all this absurd number of lines:

SQL Server - Grouped Concatenation convert rows into string - Performance - CLR4
SQL Server - Grouped Concatenation convert rows into string - Performance - CLR4

As for FOR XML PATH… It ended up getting lost and the results were not in the correct order.

SQL Server - Grouped Concatenation convert rows into string - Performance - STUFF FOR XML PATH4
SQL Server - Grouped Concatenation convert rows into string - Performance - STUFF FOR XML PATH4

Therefore, the solution that I recommend as the best of all for this type of situation is SQL CLR!

That's it, folks!
Thanks for visiting and see you in the next post.

sql server clr function stuff for xml path recursive cte convert columns rows grouped concat concatenation string how to convert concatenate columns to string

sql server clr function stuff for xml path recursive cte convert columns rows grouped concat concatenation string how to convert concatenate columns to string

Concatenate many rows into a single text string

Concatenate many rows into a single text string