Advertisement

03.29.2006 at 07:29AM PST, ID: 21793653
[x]
Attachment Details
[x]
The Solution Rating System

With so many solutions, how can you tell which solutions are most likely to help you and which ones are not? To provide you with a tool to use, we rate our solutions based on various elements that most accurately determine if a solution is a quality solution. To explain what factors affect the solution rating, here are the elements we take into consideration when formulating our solution rating.

  • The Grade of the Solution
  • The Zone Rank of the Expert Providing the Solution
  • The Number of Author and Expert Comments
  • The Number of Experts Contributing
  • The Feedback of the Community

Your Input Matters
Because of the way the system is set up, the most important variable in this equation is you. As a member of Experts Exchange, you are able to cast your vote on the quality of the solutions in regard to how complete, accurate, helpful and easy to understand each solution is. When you provide your feedback, each rating is adjusted accordingly. So, if you see a solution that has a poor rating that you think is a good solution, let us know by rating it. As you do, the rating will be adjusted and will become more accurate for other members of our site.

If you have any suggestions that you would like to make for our rating system, please ask a question in the Suggestions Zone of Community Support.

Thank you!

8.2

Bash shell script - AWK Remove blank lines from text file while stripping certain characters

Asked by cybrthug in Unix Operating Systems

Tags: , , , ,

I have a text file im trying to parse out information after a certain line with a keyword. After this keyword all the information
below it is important to me, but for some reason its stripping all the escape characters or line characters that im needing
to process the information with another script. The other script I run takes this information and makes it delimited so I can
import into mysql. While peeking at the file with cat -vet I need it to look like this:

awk '/../&&d{print}/QUANTITY/{d=1}' test1.txt > test2.txt


FILE: test1.txt ORIGINAL FILE
---------------------------------------------------------------------
TABULATION^M$
QUANTITY^M$
^M$
^M$
    11111111  GIANTS                                   222.000     TON^M$
 7777      Acme Inc.                                           6.5000        3,425.50        3,425.50^M$
 8888      Pipe Ind, Inc.                                      1.0000          527.00          527.00^M$
^M$
    22222222  BEARS                                    324.000     TON^M$
7777      Acme, Inc.                                         148.3800        2,522.46        2,522.46^M$
8888      Pipe Ind, Inc..                               120.0000        2,040.00        2,040.00^M$
-----------------------------------------------------------------------

With awk it leaves in ^M$ after every line like the following and my other script wont read it right.

test2.txt comes out like this:

FILE: test2.txt
---------------------------------------------------------------------
    11111111  GIANTS                                   222.000     TON^M$
 7777      Acme Inc.                                           6.5000        3,425.50        3,425.50^M$
 8888      Pipe Ind, Inc.                                      1.0000          527.00          527.00^M$
    22222222  BEARS                                    324.000     TON^M$
7777      Acme, Inc.                                         148.3800        2,522.46        2,522.46^M$
8888      Pipe Ind, Inc..                               120.0000        2,040.00        2,040.00^M$
-----------------------------------------------------------------------


What I need test2.txt to look like is the following, I need to remove the ^M but leave the $ at the end of each line, ALSO!
very important for me to process I need to remove the 2 blank lines before the information like you see in the original
file after the word QUANTITY. It must look like this for my other script to work properly.

FILE: test2.txt
---------------------------------------------------------------------
    11111111  GIANTS                                   222.000     TON$
 7777      Acme Inc.                                           6.5000        3,425.50        3,425.50$
 8888      Pipe Ind, Inc.                                      1.0000          527.00          527.00$
$
    22222222  BEARS                                    324.000     TON$
7777      Acme, Inc.                                         148.3800        2,522.46        2,522.46$
8888      Pipe Ind, Inc..                               120.0000        2,040.00        2,040.00$
-----------------------------------------------------------------------

Thanks in advance for any help you can offer.

Start Free Trial
[+][-]03.29.2006 at 08:16AM PST, ID: 16323104

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 7-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]03.29.2006 at 08:23AM PST, ID: 16323177

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]03.29.2006 at 08:26AM PST, ID: 16323221

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]03.29.2006 at 09:00AM PST, ID: 16323579

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 7-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]03.29.2006 at 10:00AM PST, ID: 16324229

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
[+][-]03.29.2006 at 10:34AM PST, ID: 16324612

View this solution now by starting your 7-day free trial. Setting up your free trial is quick, easy, and secure. We will return you to this solution, unlocked, when you're done.

 

About this solution

Zone: Unix Operating Systems
Tags: remove, script, shell, lines, bash
Sign Up Now!
Solution Provided By: ozo
Participating Experts: 2
Solution Grade: A
 
 
[+][-]03.29.2006 at 10:51AM PST, ID: 16324825

At Experts Exchange, members can ask their questions to thousands of technology professionals, also known as Experts. Experts compete and collaborate to answer those questions by leaving comments like this one.

Start your 7-day free trial to view this Expert Comment or ask the Experts your question.

 
[+][-]03.29.2006 at 12:37PM PST, ID: 16326151

Often, when Experts are collaborating with members who have asked questions, they will request additional information about the problem. Askers respond with an author comment like this one.

Start your 7-day free trial to view this Author Comment or ask the Experts your question.

 
 
Loading Advertisement...
20080716-EE-VQP-32