[600MRG] WSPRnet database

[Larry]

Bob,

See below:


On 31-Oct-14 7:32 AM, Bob & Jane wrote:
> Not sure I can add anything to what has already said but here goes.
>
> I downloaded the October file yesterday, so it is not the complete
> October data.
>
> As stated, there are about 6.4 million records so far and the .csv file
> is 551 MB when unZipped.
>
> I wrote a Visual Basic program to strip out all but the 600m records. I
> am down to 510,150 records and about 50 MB file size.


###  This can be done with a one line "grep" or "find" command but VB 
would certainly allow more sort options assuming one has the resources 
to deal with VB.


>
> Then, I stripped out all but USA transmitting stations. Now I am down to
> 200,487 records and 20 MB file size.
>
> Then I stripped out a few of the fields that I didn't think would be
> necessary (only stripped out 4). Of course, I still have 200,487 records
> but the file size is now 15 MB.

### This is where VB or some other high level programming is valuable, 
but once the file size is reduced with the simple commands, everything 
else could be done in a spreadsheet.


> Open Office was suggested, but an Internet search indicates that has a
> limit of 65,536 rows per sheet. 200,000 records is too many.

####  OO installed on a 64 bit windows 7 box has a one mega row limit, 
32 bit systems much less. I believe current Microsoft Office/EXCEL 
versions are the same.


>
> One possibility (which I haven't tried) is to segment each transmitting
> stations records into a separate Excel sheet. My old version of Excel
> (2000) states the limit on the number of sheets in a workbook is limited
> by available computer memory. Open Office has a limit of 256 sheets per
> workbook. This would limit the types of analysis that could be performed.


#### This is the sort of analysis that is beyond my present 
capabilities. I understand the concepts but the details of 
implementation are lost in the old age CRS dust......

I was really hoping some spreadsheet guru would come forward with a few 
simple macros and help out but it looks like all "modern" solutions are 
well beyond the capabilities of us RF hardware guys. Oh Well.

73

Larry - W7IUV / WH2XGP