Find Jobs
Hire Freelancers

dynamic CSV to SQL server import - automated using SSIS C#

₹1500-12500 INR

Mbyllur
Postuar about 6 years ago

₹1500-12500 INR

Paguhet në dorëzim
Project Requirement: I want to create a solution to load a dynamic csv file to SQL table. Following requirements need to be met: 1. The input CSV/TXT file could have any number of columns. 2. Column Delimiter of input file could be comma(,), pipe (|), tab (<tab>), basically it could be different for different file but will be standard in a single file 3. Row delimiter could be [CR][LF] or [LF] or [CR] etc 4. The solution should load all the contents of this file to a SQL server table. (sql version could be 2012/2014 etc) 5. The destination table will be created based on the column names in the header row of the input file 6. All the destination table columns should have sql data type nvarchar(max) 7. Destination table should have PAGE type compression in SQL server 8. Preferably the solution should be a SSIS package or a programatically generated and executed SSIS that loads any files into a table it creates. 9. Solution should just accept a path to the CSV/TXT file and should be able to figure out everything else automatically (number of columns, create table and then load) 10. I should be able to use this to load to SQL 2012, SQL 2014, SQL 2016 Performance requirements 1. The input files I have are very heavy 50GB, 70GB, 80GB etc, so the solution needed should be able to work with such files and also load the the data quickly 2. I already have a few solutions: a) Load csv into a single column table and then parse the column names and data to a table will multiple columns, this process is too slow, so its not acceptable b) use C# script to read header row from csv file, create a table and then load the data using C#. This process is also slow So the requirement is to have the solution as fast as a BCP task in SSIS. I am able to build a static BCP where filename is provided, column structure is static, this works fast but need is to make this dynamic. Copyright requirement: 1. Cannot use any thing that is patented to someone 2. Cannot used a paid library, SSIS addon, every thing should be developed in house 3. you can simply use C#, SSIS, script component etc.... microsoft provided features, so there is no question of using anything copyrighted Documentation Requirements 1. Everything should provide should be documented, so it can be easily reproduced by me on a new installation of SQL/SSIS/Visual Studio etc 2. Code needs to be commented, so every line to code has an explanation
ID e Projektit: 16256939

Rreth projektit

17 propozime
Projekt në distancë
Aktive 6 yrs ago

Po kërkoni të fitoni para?

Përfitimet e ofertës për Freelancer

Vendosni buxhetin dhe afatin tuaj
Paguhuni për punën tuaj
Përshkruani propozimin tuaj
Është falas të regjistrohesh dhe të bësh oferta për punë
17 freelancers are bidding on average ₹9 376 INR for this job
Avatari i Përdoruesit
Hi, I have gone through your REQUIREMENT. I can help you in this as I have much EXPERIENCE in Asp.net, C#, MVC, JSON Web services , ATOM Web services. Can we discuss the REQUIREMENT in detail? We work on different platform like:- . Website designing/ website developing(Asp.net, C#, html, css) .server side scripting language(php) .wordpress Asp.net C# MVC Wordpress Html CSS Web Development and Web Designing. JSON Web services ATOM Web services E-commerce Web development
₹10 000 INR në 20 ditë
4,9 (16 përshtypje)
5,0
5,0
Avatari i Përdoruesit
Hi! Read your project proposal about fast import of CSV data to SQL Server. This is a really interesting and challenging task for me, I really want to do this. I have a 15 years experience on MS SQL database development, administration and software development. Have a good knowledge and many completed projects on SSIS. I'd like to know some more about your performance issues with previous attempts - what time on 50GB file (for example) you consider acceptable? Please, message me to discuss this and some other details.
₹8 777 INR në 10 ditë
5,0 (8 përshtypje)
4,8
4,8
Avatari i Përdoruesit
I have 8 + years of experience of working on SSIS & SQL server based technologies. I have completed these typed of projects in the past .
₹13 333 INR në 6 ditë
5,0 (3 përshtypje)
4,9
4,9
Avatari i Përdoruesit
I have ready package to load a file and import data , i will do for all files as well if you would like to take it . please let me know
₹8 888 INR në 2 ditë
4,9 (18 përshtypje)
4,4
4,4
Avatari i Përdoruesit
Please review my profile. https://www.freelancer.com/u/vw7590795vw.html I have 8 years and 3 months experience in SQL and 5 years and 5 months experience in SSRS, SSIS. Please let me know more details and when we will start. Sky pe: jikenjd
₹6 500 INR në 5 ditë
5,0 (23 përshtypje)
4,2
4,2
Avatari i Përdoruesit
A proposal has not yet been provided
₹13 333 INR në 10 ditë
4,4 (9 përshtypje)
3,7
3,7
Avatari i Përdoruesit
A proposal has not yet been provided
₹12 500 INR në 3 ditë
0,0 (0 përshtypje)
0,0
0,0
Avatari i Përdoruesit
A proposal has not yet been provided
₹9 288 INR në 5 ditë
0,0 (0 përshtypje)
0,0
0,0
Avatari i Përdoruesit
I have worked similar project where i have to load hedis measures. It is a healthcare quality measure project.
₹6 666 INR në 3 ditë
0,0 (0 përshtypje)
0,0
0,0
Avatari i Përdoruesit
I am currently a Business Intelligence Consultant, where I work with ETL and also with reports and Dashboards. With ETL I work with SSIS, where to read the data of a text file to load in a table of SQL, to me it is quiet, probably what will give more work is the question of the file being mutito heavy, but already I worked with tables where they had 100 million records. I am available.
₹8 888 INR në 10 ditë
0,0 (0 përshtypje)
0,0
0,0
Avatari i Përdoruesit
Hi... It's Easy to do this work in normal form when we insert data from any type of file like (CSV,text,docx) to database....But when we insert 50GB of data it mean it carry lots of rows ... and lots of time... so don't worry I will do this work in proper way ... Now I tell something about myself I'm Akash and I'm software developer (.net Specialist) I currently working on CBS(Core banking solution) project . in that project Also have same cases when we insert lots of customers data from Database to file and file to Database... so I have good experience for how we do your work... I'm already done this job.. Thanks..
₹8 888 INR në 3 ditë
0,0 (0 përshtypje)
0,0
0,0

Rreth klientit

Flamuri i INDIA
India
0,0
0
Anëtar që nga shk 5, 2013

Verifikimi i klientit

Faleminderit! Ne ju kemi dërguar me email një lidhje për të kërkuar kredinë tuaj falas.
Ndodhi një gabim gjatë dërgimit të email-it tuaj. Ju lutemi provoni përsëri.
Përdorues të regjistruar Punë të postuara
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Po ngarkohet shikimi paraprak
Leja u dha për Geolocation.
Seanca e hyrjes ka skaduar dhe ke dalë. Hyr sërish.