Thursday, April 7, 2011

[Powershell] Basic SQL query exported to CSV


Invoke-Sqlcmd -query "select getdate(), getdate()+1" -serverinstance "yourservername"|Export-Csv c:\temp\testps2.txt –notypeinformation

or, broken down by line so you can see all of it...


Invoke-Sqlcmd
-query "select getdate(), getdate()+1"
-serverinstance "yourservername"
|Export-Csv
c:\test\test.txt
–notypeinformation

-query: the query.
-serverinstance: server name
-notypeinformation: removes the “#TYPE System.Data.DataRow” line at the top.

And if you don't want a header row... you have to use a different export process, and then tell a different process to iterate through the array and write to disk. Really, guys? Too hard to add a -noheader option?

(and all this is on one line; you can use a ` to split it across lines.

Invoke-Sqlcmd -query "select getdate(), getdate(); select getdate()+1, getdate()+1" -serverinstance "yourservername"|ConvertTo-Csv -notypeinformation -outvariable outdata; $outdata[1..($outdata.count-1)] |ForEach-Object {Add-Content -value $_ -path "c:\temp\test.txt"}

or
Invoke-Sqlcmd -query "select getdate(), getdate(); select getdate()+1, getdate()+1" `
-serverinstance "ftw-sv-db-03"|ConvertTo-Csv -notypeinformation -outvariable outdata;`
$outdata[1..($outdata.count-1)] |ForEach-Object {Add-Content -value $_ -path "c:\temp\test.txt"}
(then hit enter again to tell it you're done for realsies)

Oh, and it for some reason outputs the full file to console, but saves what you want to a file.

Wednesday, April 6, 2011

[Powershell] Basics to run a SQL query

Putting this here for when the new guy starts. The learning curve can suck at certain points, like the installer. See my other post about it. Grr.


  • Install Powershell 2
  • Install SQL Server 2008 Feature Pack: Powershell Extensions http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=ceb4346f-657f-4d28-83f5-aae0c5c83d52
  • Install SQL Powershell Extensions: http://sqlpsx.codeplex.com/
  • Add this line to My Documents\WindowsPowerShell\profile.ps1: "add-pssnapin SqlServerCmdletSnapin100; add-pssnapin SqlServerProviderSnapin100;" (no quotes)
  • Start Powershell and see if it works:
    Set-ExecutionPolicy RemoteSigned (or
    Invoke-Sqlcmd -query "select getdate(), @@version" -serverinstance "yourservername"|Export-Csv c:\testps.txt –notypeinformation

[Maintenance] Checking age of statistics

Simple stuff, saved here in case anybody needs it

SELECT objects.name AS object_name, indexes.name AS index_name,
STATS_DATE(indexes.OBJECT_ID, index_id) AS StatsUpdated
FROM sys.indexes
INNER JOIN sys.objects ON indexes.object_id = objects.object_id
--WHERE objects.OBJECT_ID = OBJECT_ID('dbo.yourtablename')
GO

Tuesday, March 22, 2011

[Replication] Nasty bug and reinitializing without a snapshot

So, there's a bug in SQL Server 2005 SP2 _and_ SQL Server 2005 SP3 (fixed in SP2 CU12 and SP3 CU3), where a commonly-used replication SP can inadvertently reset your subscription, and you get the dread "The initial snapshot for publication...is not yet available" error. In my case I couldn't easily snapshot the data across, so a workaround is to sync the data, then recreate the subscription, specifying that it does not need to be initialized.

Bug: MS KB967192
http://support.microsoft.com/kb/967192/EN-US

Fix:
http://technet.microsoft.com/en-us/library/ms151705.aspx

Essentially, what you need to do is drop the subscription, sync the data, then recreate the subscription, making sure to uncheck "Initialize" (if using SSMS).

Thursday, March 10, 2011

[Backups] Verify your backups physically exist

We've been using Rodney Landrum's SSIS package to monitor our environment (As Seen In SQL Server Mag). We're running the old version, which doesn't deal AT ALL with servers not being available. (I assume the latest version does, but haven't had time to check.)

And I came across an issue recently - missing backups. Because of the various retentions set via our backup jobs, we would occasionally have a file vanish. Eek!

So, cue this code. It'll grab the most-recent backup for each server/database, and make sure the file physically exists. It doesn't check the veracity of the backup, just that there's a file there. It also uses xp_fileexists, an undocumented (and therefore it can change - though it's been the same since SQL Server 2000) SP.

If you don't use Rodney's code, you can still use this, but it'll be a _little_ more work. Take the below code, have it run on each machine and dump into a central table (I'll leave those details up to you), then run the second set of code against it.


SELECT server_name,
database_name,
physical_device_name,
backup_start_date,
'FULL' as backup_type
from msdb.dbo.backupmediafamily
inner join msdb.dbo.backupset
on backupset.media_set_id = backupmediafamily.media_set_id
where backup_start_date > getdate()-14
and physical_device_name NOT LIKE 'VDI_%'
and physical_device_name like '%BAK' --or whatever your backups are named.


And here's the full code:


SET NOCOUNT ON
USE DBA_Rep
if object_id('tempdb..#backup_list') is not null
drop table #backup_list;
CREATE TABLE #backup_list (id int IDENTITY, server sysname, database_name sysname, physical_device_name VARCHAR(520), backup_start_date DATETIME, file_exists BIT)
DECLARE @minid INT, @maxid int
DECLARE @does_it_exist INT
DECLARE @filename VARCHAR(500)

--using dba_rep's copy that Rodney Landrum's SSIS code pulls, get a list of the most recent backup for each db in past 2 weeks
INSERT INTO #backup_list
( server ,
database_name ,
physical_device_name,
backup_start_date
)
SELECT Backup_History.server,
Backup_History.database_name,
Backup_History.physical_device_name,
Backup_History.backup_start_date
FROM Backup_History
INNER JOIN
(
SELECT server, database_name,
MAX(backup_start_date) AS max_start_date
FROM Backup_History
WHERE backup_type <>'LOG'
AND backup_start_date > GETDATE()-14
GROUP BY server, database_name
)most_recent
ON most_recent.SERVER = Backup_History.Server
AND most_recent.database_name = Backup_History.database_name
AND most_recent.max_start_date = Backup_History.backup_start_date
AND Backup_History.physical_device_name NOT LIKE 'SQLsafe%'
AND Backup_History.backup_type <> 'LOG'

--Fixing the names of local backups so that we can get them over the network.
UPDATE #backup_list
SET physical_device_name = REPLACE(physical_device_name,LEFT(physical_device_name,2), '\\' + LTRIM(RTRIM(server)) + '\' + LEFT(physical_device_name,1) + '$')
WHERE physical_device_name LIKE '%:%'

SELECT @minid = MIN(id) , @maxid = MAX(id) FROM #backup_list

--Walk the list, checking each file and updating the table
WHILE @minid < @maxid
BEGIN
SET @does_it_exist = 0

SELECT @filename = physical_device_name
FROM #backup_list
WHERE id = @minid

EXEC Master.dbo.xp_fileexist @filename, @does_it_exist OUTPUT
UPDATE #backup_list
SET file_exists = @does_it_exist
WHERE id = @minid

IF @minid % 10 = 0 PRINT @minid

SET @minid = @minid+1
END

SELECT * FROM #backup_list WHERE file_exists = 0

Thursday, March 3, 2011

[Tools] Convert PDF to text

Courtesy of the interwebs:

"That depends. If it is an image in the PDF, you're out of luck.
Otherwise (if it is a PDF containing text) you can do the following, all in one line, assuming osx has the strings command and a perl interpreter:"


strings filename.pdf | perl -ne '$line=$_; $s=$line; $w=""; while ($s =~ m/(\w+)(.*)/){print $line if ($w eq $1); $w=$1; $s=$2;}'

Wednesday, February 23, 2011

[Tips] Create a comma-separated list in one query using COALESCE

Clever, clever. Wish I could take credit for it.


-----------------------------------
--Creating a comma-separated list--
-----------------------------------

DECLARE @EmployeeList varchar(100)

SELECT @EmployeeList = COALESCE(@EmployeeList + ', ', '')
+ CAST(id AS varchar(15))
FROM temp_td_200704

SELECT @EmployeeList

Friday, February 18, 2011

[SSAS] Quick notes on logging queries run against the cube

There's a lot of details out there about OlapQueryLog, but here are the 3 things I ran into setting it up:


Tips and tricks from setting this up:
  1. Do it on whatever server hosts the cube, since it will determine what version of the SQL driver it needs.
  2. Event Viewer will tell you the errors you’re having, be they permissions, bad version of the SQL driver, etc.
  3. The key settings:
    • Log\QueryLog\CreateQueryLogTable = true
    • QueryLogConnectionString (click and set it)
    • QueryLogSampling is "every X queries, save the query to table". So the default means every 10th query gets saved.

Thursday, February 17, 2011

[Trick] Eliminating "arithmetic overflow error" that aren't in the result set.

Ran into a problem where the SP would always kick out the standard overflow message:

Arithmetic overflow error converting numeric to data type varchar


As it turns out, the problem was data that exists in the data set, but not in the results set - the WHERE clause eliminated it. What made it even harder to troubleshoot was that because that was the issue, we could duplicate it by running the SP - but not by copying and pasting the code. No matter what options you used - ARITHABORT, ANSI_WARNINGs, etc, it would run successfully, skipping over the bad record (since it was excluded via the WHERE clause).

As is pointed out in "Defensive Database Programming with SQL Server" by Alex Kuznetsov (WELL worth getting, and Red-gate offers a free PDF), you can't guarantee the order things get evaluated in.

So one way around it: take your query that is failing in the SELECT clause. Find a unique key that you can pull from your data set. Copy/paste the entirety of the FROM/WHERE, and SELECT only this key into a temp table. Now go to your full query and INNER JOIN this temp table.

Another potential way around it: remove everything from the where clause and keep it in the JOINs.

[Code] Stripping low-ascii out of a table, slow way

I'm working on a CLR to do this considerably faster, but here's what I have for now. The purpose of this is to remove dirty data from an upstream feed. We can't make XML with it - the code that creates the XML chokes on it. Note that this isn't terribly fast, because we're brute-forcing it and having to loop through (31 * number-of-fields) times. The CLR should do it in one pass per field.


--CREATE PROCEDURE [dbo].[strip_low_ascii]
--as
DECLARE @columns TABLE (id INT IDENTITY, column_name sysname)
DECLARE @odd_ascii CHAR(1),
@ascii_value int,
@column_start smallint,
@column_end SMALLINT,
@sql NVARCHAR(4000),
@field_name sysname, --to allow us to loop through all fields
@table_schema sysname,
@table_name sysname
SET @ascii_value = 31 --31 aka 0x1F. Space is 32/0x20. 0-30 is our "low-ascii" range
SET @table_schema = 'dbo'
SET @table_name = 'yourtablename'

CREATE TABLE #list_of_id (id bigint primary key)

INSERT INTO @columns
SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = @table_schema
AND TABLE_NAME = @table_name
AND DATA_TYPE IN ('char','varchar','nchar','nvarchar')

SELECT @column_start = MIN(id), @column_end = MAX(id) FROM @columns

WHILE @ascii_value >= 0 --look for all low ascii
BEGIN
SET @odd_ascii = NULL
SET @column_start = 1
SELECT @odd_ascii = CHAR(@ascii_value)

WHILE @column_start <= @column_end
BEGIN
TRUNCATE TABLE #list_of_id
SET @field_name = NULL
SET @sql = NULL
SELECT @field_name = column_name FROM @columns WHERE id = @column_start
SELECT @sql = 'insert into #list_of_id
SELECT id FROM ' + QUOTENAME(@table_schema) + '.' + QUOTENAME(@table_name) + '
WHERE insert_datetime >=CONVERT(CHAR(8),GETDATE(),112)
AND ' + @field_name + ' LIKE ''%'' + @oddascii + ''%''
IF @@rowcount >0 --( select count from @list_of_id )
update ' + @table_name + ' set ' + @field_name + ' = REPLACE(' + @field_name + ', @oddascii, '''')
WHERE insert_datetime >=CONVERT(CHAR(8),GETDATE(),112)
and id in (select id from #list_of_id)
AND ' + @field_name + ' LIKE ''%'' + @oddascii + ''%'''


--PRINT @sql

EXECUTE sp_executesql @sql, N'@oddascii char(1)', @oddascii = @odd_ascii

SET @column_start = @column_start + 1
END

SET @ascii_value = @ascii_value-1
END