Slow scans, inefficient system resource usage with Silverfast 8.0.1r13 DEMO and V700 [Archive]

View Full Version : Slow scans, inefficient system resource usage with Silverfast 8.0.1r13 DEMO and V700

architorture

21-Jul-2012, 11:47

Hi everybody,

I have posted this question/complaint on the silverfast forum, but that seems to be full of people complaining about how slow the response is and how unhelpful! I also posted this in a couple groups on flickr, so apologies if you come across this in multiple places. I thought I'd post here and see if anyone has experienced the same thing. Thanks for your help!

The post:

I am a current user of the bundled Silverfast SE that came with my Epson v700 scanner. My software is updated to 6.6.2r3.
I recently downloaded the AI Studio 8.0.1r13 DEMO so that I could test the new version 8 and evaluate whether I should upgrade.

I think the image quality/tonality is slightly improved in 8, and would like to upgrade from my version SE anyway so that I can save to 48bit color with processing (NegaFix). However, in testing the DEMO of version 8, I have found it completely unusable for my situation.

I scan MF and LF film at 4800spi, immediately downsampling to 2400dpi in Photoshop. I prefer to scan at double resolution to eliminate grain aliasing and reduce noise in the scanned image, taking advantage of the effects of the 2x2 pixel averaging. This does create very large files, but with version 6.6.2r3 the scan time is reasonable and the workflow acceptable. Scanning a 4x5 negative in 48bit HDR mode takes under 10min, from start to saved file (2.5GB).

I tried to scan a 6x7cm color negative with the Demo version of SF-8, but found it incapable of doing so reasonably. A 4800spi 48bit scan of the 56x68mm image took 45 minutes! (755MB). I also tried other settings: 4800spi 48->24bit took 49 minutes with 8 (410MB), compared to 7 minutes with version 6.6 (406MB). I am able to scan at appropriate speeds with version 8 at lower resolutions - a 3200spi 48->24bit scan took only 4min (184MB), as I would expect. I also tested at 2400spi and 1200spi, 48bit, without problems, and also successfully scanned only a small portion of the negative at 4800spi 48bit (76MB).

When scanning the entire negative with version-8 at 4800spi and 48bit, I can hear the drive motor in the scanner run for only about 5 out of every 20 seconds. Most of the 45 minutes it is not actually scanning! It is in fact constantly writing to the HDD, while the memory usage in the windows task manager graph goes up-and-down like a heartbeat (but never above 40% of the total). The silverfast 8 process never uses more than 340MB of memory, despite between 4 and 6 GB being "available" according to the task manager. It also seems to create four 700-800MB files in the temp directory. In contrast, when scanning with my current version 6.6, the memory graph is a completely flat line, the HDD is idle until the scan is over and the file is saved, and everything runs smoothly. If helpful I can upload or send screenshots I recorded of the task manager while scanning in these different situations.

I know the immediate response is that it would be a memory issue but the fact that I can scan, with the same settings, a file that is 3.4 times larger, on the same system, using the older software, eliminates this explanation in my opinion. Poorly written code and inefficient system resource usage in version 8 is the only potential difference.

For reference, my computer: windows 7 professional x64, service pack 1; intel core-2 quad Q9550; 8GB DDR2-800 RAM; traditional HDD with 116GB (50%) free space.

At this point I would like to purchase the upgrade for a version of SF-8, but cannot do so because the software is wholly non-functional for my needs.

Thanks for your help - I look forward to any responses.

****END****

p.s., I also have Vuescan Pro 9, but find it intolerable - people say silverfast has a bad UI!
pps, none of this is an issue with black and white, because i don't need negafix for that - just scan in 48bitHDR mode and adjust in photoshop. Another win for B&W!

Brian C. Miller

21-Jul-2012, 12:21

Huh. I never thought that I could help out with digital photography problems, but this happens to be one of my areas of actual professional experience.

Yes, you have a software and system performance problem. However, you're not going to like the only solution to Silverfast 8. One way or another, your are going to have to massively reconfigure your system.

You have one hard drive in your system. Windows Server can move the temporary file storage to another hard drive, which could be a striped array, but not Pro. You are going to need to load Windows onto a striped drive array, and this can only be configured in your BIOS setup. Alternately, see if there is a setting in Silverfast 8 to change the location of where it writes the temporary files. If this can be changed, then your files can be written to an striped array, which is simple to add.

architorture

21-Jul-2012, 12:38

hm, thanks for the quick response Brian!

I actually have two drives in my case - a 250GB system drive (117GB free), and my 1TB "Data" drive (158GB free), where the scans get saved to in the end. I also have two external backup drives, but those are not turned on generally.

There is an option in SF-8 to change the temp file location (not available in 6.6). I'll try moving it to the Data drive, and report back on how that affects things.

For what it's worth, I'm probably going to get an SSD and re-install the OS with that as my system drive. It would certainly speed things up with SF-8, but it still seems like it shouldn't need speeding-up, based on how smoothly SF-6.6 runs (with bigger files even).

I'll report back soon -

Brian C. Miller

21-Jul-2012, 12:47

If your data drive is faster (check the specs) than your system drive, that's good. What is limiting you is how fast the data can be written to the drive, i.e., the sustained data rate. You probably need two (or more) drives with high spindle speeds (like 10K), and then make a striped RAID 0 (not spanned) drive array out of them. What this means is that, at the same time, the data is written to all the drives in the array. So in a two drive array you get 2x the performance, three drive is 3x, etc.

When a drive is used for nothing but temporary data, then you don't have to worry about performance problems with fragmentation. Fragmentation is when the drive data is written all over the disk, as file space permits, instead of a contiguous file. This is why in server operating systems the temporary data is kept in its own partition, and usually on a seperate drive.

architorture

21-Jul-2012, 13:37

Ok, so, I tired again with the Temp files on the 1TB data drive and total scan time was 38 minutes. Improved, but still terrible. Behavior was identical as before.

I understand what you're saying - the scan process is actually two phases: the first 19 minutes is actually scanning, writing to the disk continuously as it goes, stopping and starting. The final 19 minutes is just "processing" according to Silverfast. There is no disk usage, but it sits with constant 50% CPU usage and very little memory activity/use. So a faster disk (ssd, raid 0, whatever) would shorten the time it takes because it would write the data faster. (with the old 6.6 "scanning" takes about 8min, "processing & saving" 1.5 minutes, and that's with a 4x5 negative at 4800spi 48bit).

I don't think fragmentation is an issue - there is so much free space, and I also don't hear the drive heads moving around constantly like they do during random read/writes.

I'm sure you're right that a RAID 0 array would make it faster, but it still seems silly to me that that is required. It just seems like poorly coded software. Why would it want to write to a disk (even ift's RAID 0) when there are 5GB of RAM sitting there that it can access at least ten times faster than any disk!? Why would they configure it to not use the RAM, and wait for the disk write operations instead? And also, why would a "completely re-written" piece of modern software not utilize more than 2 cores (or 50% of 4)? It's all the more frustrating because they seem to have understood these things when they made the old version 6.6.

Thanks again for your help Brian.

LSIGregoire

24-Jul-2012, 00:03

Jody_S

24-Jul-2012, 05:29

You're running a 32-bit version of Silverfast 8 Demo. It is limited to 2GB RAM use, because it's the 32-bit version, so it is using the HDD as a scratch disk on the 40+ minute scans. I don't know if they make a 64-bit version of the program, but if they do that would fix your problem. I expect the problem occurring only at higher resolutions is because you're scanning above the optical resolution of the scanner, so it is extrapolating to create the larger file, which is very processor-intesive, and perhaps memory-intensive if it's a good algorithm that is sampling more than a 2x2 square to do the extrapolation.

I assume, with those speeds, you're not doing any dust/scratch removal in-scanner?

photobymike

24-Jul-2012, 11:35

I had similar problems with a PC and windows Vista. I was going to upgrade to windows 64. It required that i buy a new computer. Here is what i did to fix the limitation problem with total costs kept in mind.
1- bought used power mac G5 $250
2- upgraded memory from 4 to 8gb $125
3- downloaded MAC epson drivers
4- Got a MAC version of Silverfast
5- connected G5 MAC to router and to net drive (read write both MAC and PC)

I really make scans faster than i could ever make on a PC; noticeably faster and lager files. And much to my surprise this bargain computer is faster than my i7 PC in photoshop surfing the web ect... and its while i am scanning. That was 2 years ago and my 8 year old MAC is running strong as i am typing this note. Still faster than my PC which my kids are using now to play games and surf. I dont want to start a war here. There is a lot of passion between PC and MAC people... I really dont care about that i just wanted a cost effective solution. Also Unlike my PC laptop with win 7, my MAC G5 has never ever ... not even once locked or given me an error....

so you spent some major bucks to for your scanner why not get a computer dedicated for scanning only... it would cost about half of what you spent on your scanner. I dont know why people think there computer needs to do everything compatible with everything... One size does not fit all solutions sometimes is not the most efficient. I have since upgraded to an intel based MAC. Photoshop screams on my used MAC... 10 seconds to open and once open, it stays in memory and reopens instantly. Changes happen instantly ...

Well bottom line think about an older used MAC G5 or newer to run your scanner. The files it makes are compatible with your PC, including your PSD Photoshop files.....

Please dont turn this into a MAC vs PC war... This what i did to solve a specific performance problem with Epson and a PC. I have maintained a separate scanning station for about 2 years now that i am very happy with... think outside the box to get the solution that works for you.

Jody_S

24-Jul-2012, 12:22

photobymike

24-Jul-2012, 13:04

Shorter version: Bill Gates sucks.

LOL LOL LOL i was hoping to be constructive and help the poor guy. The real problem is that Windows tries to be all things to all people. When the reality is individual computers and software have strong points and weak points. Kinda like people

Which begs the question .... why is he scanning at such hi resolutions.... there is a point where more is not better, it just a waste of time ..... for printing 16x20s that look gorgeous on say a 6x7 120 neg which i do all the time i scan at 3200 and sometimes 4800 ... thats huge.... There comes a point where there is diminished return on resolution and sharpness. I never use 48 bit for saving or scanning My printer (epson 4800 and R3000) does not have the color space or the resolution to print that.... when you start the printing process it takes all this picture information and translates down into what the printer can print..... Total waste of time to scan at hi resolutions higher than what is needed... There is an optimum for resolution and sharpness applied... if you go beyond this it is a waste of time and your picture looks bad. For instance a scan ment for the screen needs little sharpness applied and a scan that is not hi rez.. most screens display at 72 pixels mine does 96 pixels.... it just takes practice to find the optimum settings

SergeiR

24-Jul-2012, 16:10

To be honest, Silverfast isn't stellar for me either, and i do run it on Lion OSX, with 2.7Ghz quad core and 16g of ram. Same stuff done in Vuescan goes fairly fast. Silverfast - i had never managed to scan a single sheet of 8x10 with it. Only 120 film. It just never ever finishes processing.

architorture

25-Jul-2012, 01:29

LSIGregoire:
I'm really glad you guys from Lasersoft have responded here and on flickr - I think I'll just reply here. Thanks for your work on this.

I had most recently rebuilt and re-installed all the software on my computer about 2 years ago - with Windows 7 x64. So I assumed that I had installed the proper 64bit drivers at that time (and I think I most likely did). Nonetheless, the idea had not occurred to me before your suggestion, so I went to Epson's site and grabbed the most recent x64 driver, uninstalled the scanner through device manager, and installed the new driver.

Tried to scan again with all the same settings as previously - total time 36 minutes (16 + 20). So, a slight difference, but not much, and possibly just normal variation. All the behavior I described above was still the same - writing to the disk continuously, not using available RAM, not using the multi-core processor fully, etc.

Is there a log-file or something that I could send to you that would be helpful in diagnosing the issue?

Jody_S:
I am NOT running a 32-bit version of Silverfast 8 Demo. The behavior I'm experiencing is exactly like what I might expect if it were 32-bit, but I am not (the exe is "Silverfast (64 bit)" - I've attached a screenshot of what it looks like in the start menu.

Thanks for mentioning iSRD - that is not activated.

Re: mac/pc, new computer, etc.
That's just irrelevant and not helpful.

General:
IMO, the point of 48bit is not for input/output but for the adjustment process in-between. As good as the scan may be, it will need some selective curves and tonal adjustments in Photoshop to realize my intent, which should be done in 16bit/channel mode. And as far as scanning for web/the screen - that's obviously not the point.

Getting into LF with 4x5 scans (at reasonable 2400dpi) was the final incentive I needed to move to an SSD - so as of yesterday I have done so. I have not yet had a chance to test silverfast again with the SSD - It's on my list for tomorrow. I'll return the cache location to the default, so that it uses the SSD. I'm sure it will be faster, but it does not address the underlying condition/circumstances of system resource usage by the software.

SergeiR: are you using version 8?
Has anyone else actually tried silverfast version 8? or are you using older releases?

I'm glad there is such an enthusiastic response to the topic on the forum! I hope it leads to a quick and positive resolution.
Thanks everybody. :D

Jim Cole

25-Jul-2012, 05:27

Do you have the same issues scanning with Epson Scan?

SergeiR

25-Jul-2012, 07:16

SergeiR: are you using version 8?
Has anyone else actually tried silverfast version 8? or are you using older releases?

Hmm.. i have whatever is latest one i think, b/c i got uprgade in May. But i will check if its 8 and not 7 once i am back at my processing desk this evening.
Like i said - it worked ok for 120.. Not brilliant, but not bad either. But when i tried 8x10 just few days ago, b/c i thought i would love to have some IR cleanup that Vuescan doesnt do on 8x10 transparency mode - poor thing just took forever to get through step 1 , and then hang on step 2. I think i shut it down about half hour later after getting to that step, b/c it was just too painful and scanned it in Vuescan.

Oh and i do have SSD..

photobymike

25-Jul-2012, 10:04

Sounds like you are an expert on this scanning thing... i would be interested in how this turns out... please keep us updated..... The longest my scanner takes on my largest images is not longer than 8 to 10 minutes. 36 minutes seems like a lot.. Well the v750 is a faster scanner than the v700...per Epson website...... I have scanned 6cmx7cm at 12800 to see how long and what file size. 600meg was just way to big.....

architorture

25-Jul-2012, 15:51

Do you have the same issues scanning with Epson Scan?

No. not at all.
I don't use epson scan normally, but I tried it by quickly going in and doing a scan in "professional mode" with the configuration options pretty much on default, but trying to mimic the adjustments applied in Silverfast. It displayed none of the symptoms I described above - infact it deemed fit to make use of the available RAM: the epson scan process maxed-out at ~900MB physical memory used, and never accessed the HDD until saving the output .tif file. I'd say the epson software is well written!
The 4800spi, 48bit, color negative scan took 2 minutes 37 seconds. That's efficient. Obviously, the quality is not quite up to the silverfast scan - but it demonstrates that the issue is software related not system induced, IMO.

Sounds like you are an expert on this scanning thing... i would be interested in how this turns out... please keep us updated..... The longest my scanner takes on my largest images is not longer than 8 to 10 minutes. 36 minutes seems like a lot.. Well the v750 is a faster scanner than the v700...per Epson website...... I have scanned 6cmx7cm at 12800 to see how long and what file size. 600meg was just way to big.....

Are you using version 8?

I wouldn't say "expert" - I just have a strong case of what some might call nerdiness and obsessiveness about some things...
I felt this would be useful to others, that's why I posted on here, as well as at the silverfast forum (they did not allow my thread there, but instead opened a support ticket).

I mentioned that I've installed an SSD - I actually have it configured at the moment so that I can boot into the OS on either my old HDD, or the new install on the SSD. For comparison's sake, I'm still booting into the old OS image.

I went through and removed the epson software/driver again, defragmented all my harddrives, reinstalled the epson driver, cleaned the registry and temp files... etc. just to be sure that none of that was causing the problems.
Result: exactly the same as before: 38 minutes total (18 "scanning" + 20 "processing").

I then changed the preferences in Silverfast so that it placed the cache on the ssd (not the OS drive).
Result: much faster, but still the same underlying, inefficient behavior - 22 minutes total (~2 "scanning" (matches the speed of Epson Scan software) + 20 "processing").

One might think that that's the optimum case; but, looking more carefully at the system usage in Resource Monitor I can see that the Silverfast process maxes-out at exactly 25% CPU usage during the entire 20 minute "processing" stage. Since I have a quad-core processor, that behavior indicates that Silverfast still can only run as a single-threaded process (100% in one thread divided by 4cores = 25%). That is very disappointing. needless to say the processing time would be much less if it used all the available cores...

I also do not think that using the SSD for the cache is a proper solution - that is exactly the kind of repetitive I/O behavior that is recommended against for SSD drives, given their limited read/write lifespan.

So, in sum, Brian's initial recommendation is probably the best way to deal with the poor system usage - a RAID 0 array should be fast enough not to limit the "scanning" portion (the Disk usage maxed-out at 245MB/s total I/O), and it avoids unnecessary operations on the SSD. Unfortunately, it does not address the software's apparent lack of multi-thread support, or the myriad other issues I'm experiencing with it.

22 minutes still seems beyond the acceptable range to me. The times quoted by LSIGregoire are exactly what I would expect with properly functioning software given what I'm seeing: ~2 or 3 minutes "scanning" plus ~5 or 7 minutes "processing" (20/4 or 20/3).

SpeedGraphicMan

25-Jul-2012, 16:46

I had the same problem! Switched to VueScan... Never looking back!
http://www.hamrick.com/

P.S. Look to see if your scanner is outputting an exact size scan (same dimensions as your film) or if it is interpolating it up to a bigger size.

SergeiR

25-Jul-2012, 18:24

just checked - mine is 8.0.1r8, couldn't be bothered to load r13, to be honest.

And 48->24bit mode, 2400 dpi..
Also, as someone who been in software design for 26+ years, i must add that its GUI is amongst most unintuitive ones i ever saw. And i did code on IBM380 and PL/M, mind you...

Bill Koechling

25-Jul-2012, 20:20

Also, as someone who been in software design for 26+ years, i must add that its GUI is amongst most unintuitive ones i ever saw. And i did code on IBM380 and PL/M, mind you...

I agree with you but I've made my peace with Silverfast (I'm still using V.6 on my v750). I get very high-quality scans with it in spite of the GUI. It took me a full year to really figure it all out. I'm not yet inclined to go to v. 8. (By the way, I wonder why they bypassed v. 7?)

photobymike

25-Jul-2012, 20:25

There is a point of diminished return on quality with resolution and color with the Epson scanner. Experience will teach you this over many thousands of scans and 5 Epson scanners v500, v600, v700... v750 < times 2 ... wore the first one out. How big and what depth depends on what you are going to use the output for.

Also some PCs have legacy hardware architecture that make it imposible to go beyond memory limitations. There is some operating software and hardware that this is not an issue. Windows 7 with a full 64bit PC mother board breaks this barrier. The Apple MAC does this out of the box. My wife has a Apple Mac Mini that i upgraded to 4 gig memory. She uses the v600 to make just awesome scans from her family photos.

My experience with computers Heathkit- 82thru 88 ...88thru 92 had my own computer store.. 92 thru 08 AOL Time Warner ..systems design, soft developer ...Certs in Novell CNE...Microsoft ...routers Cisco ...Computer Security ... Now am working on Apple repair cert. I do not know everything but i know people who do... am not really a noob at this..

if your money supply is abundant yea work on getting the PC to do what you want.... but even if i had an abundance of cash, i would not put all of your eggs in one basket

If you do not want to upgrade the PC either hardware or operating system ... which can get costly i was just suggesting that a separate scanning computer. It worked for me cheaper than upgrading the PC. Many people think or want one box to do it all... They get laser focused on the task at hand that really they need to step back and think about what they are doing. "There is a better way grasshopper."

bottom line
my solution was a cost effective way fix the issue just like you have, and i did this quickly and it was extremely easy on the wallet. I solved your problem with about 300 shekels and less than an hour work

I agree with you but I've made my peace with Silverfast (I'm still using V.6 on my v750). I get very high-quality scans with it in spite of the GUI. It took me a full year to really figure it all out. I'm not yet inclined to go to v. 8. (By the way, I wonder why they bypassed v. 7?)

Ditto

architorture

25-Jul-2012, 21:18

Also some PCs have legacy hardware architecture that make it imposible to go beyond memory limitations. There is some operating software and hardware that this is not an issue. Windows 7 with a full 64bit PC mother board breaks this barrier.

Mine falls into the latter category - I know, I built it myself.

There is a point of diminished return on quality with resolution and color with the Epson scanner. Experience will teach you this over many thousands of scans and 5 Epson scanners v500, v600, v700... v750 < times 2 ... wore the first one out. How big and what depth depends on what you are going to use the output for.

I understand the situation I'm describing is a rather extreme circumstance - only to be used for certain reasons. But it is something I would like the ability to do, something that I CAN do with 6.6 (minus the 48bit output because I never upgraded the bundled SE version before Silverfast transitioned to version 8) or Epson Scan on the same exact hardware/OS, and something I consider a requirement that must be met in order to purchase a version 8 upgrade.

As far as the silverfast GUI SergeiR and Bill, I'd rather not to turn this into a negative thread disparaging the software - I don't intend to be unnecessarily negative about the software when I'm trying to get their help. The only reason I'm putting myself through all of this is because I do think 8 is an improvement overall.
And anyway, I wouldn't say the Vuescan GUI is anything to write home about either.

I think I may just see if I can get them to sell me a license for 6.6 AI Studio. It is no longer available on their website as far as I can tell.

architorture

25-Jul-2012, 23:18

SergeiR, since you seem to be the only other one with Silverfast 8, do you think you could try scanning with similar settings as me and take a close look at the system resource usage while it is running? I would be very curious to know if the underlying behavior is the same on your system i.e. writing to disk during scan, silverfast process not utilizing more than ~400MB RAM, CPU running as only a single thread during "processing." I think it would be very useful for me as well as the Lasersoft support technician I am emailing with to know whether others experience the same symptoms.

SergeiR

26-Jul-2012, 04:46

Aye, i can try. I think i got some 6x9s in colour. Just tell me exact settings you want to try.

LSIGregoire

26-Jul-2012, 06:08

LSIGregoire:
Is there a log-file or something that I could send to you that would be helpful in diagnosing the issue?

Yes, there is --> C:\Users\username\AppData\Roaming\LaserSoft Imaging\SilverFast 877820

Please mail it as an attachment to support<at>silverfast.de and write "Att: Largeformat forum post" in the subject line - that will put directly into my mailbox. Thanks for that.
When trying to reproduce your scanning times I scanned a 6x7 cm slide in 48-bit HDR mode @4800dpi (for time comparisons) into a Tiff. No rotation, no Multi-Exposure; no iSRD, no SSD disks :-). My machine is a "regular" Win7even/64bit PC from our testing lab.
My measured times were around 12 minutes. The actual scanning times were around 5 minutes and the rest is for processing and saving.

Your LogFile should provide us with helpful information to get to the root of this problem. You will hear back from me shortly after I received the LogFile.

Kind regards,

LSI Support

architorture

26-Jul-2012, 10:20

Aye, i can try. I think i got some 6x9s in colour. Just tell me exact settings you want to try.

Image dimensions are 2.75 x 2.24
4800dpi
48bit (not 48bit HDR)
tif output

NegaFix:
Fuji, Pro, 160 S 6x6
CCR
Exposure 0.0
(in the expert options, "expansion" the Auto box is unchecked before expanding the scan frame outside of the image area - I scan with some black border around the actual image).

AutoCCR:
Portrait mode

Histogram:
all sliders at 0 or 255, as appropriate, except the mid-point slider at +9

Gradation:
Midtone +9 (same as in histogram above)
Constrast: -10

I also have 4 neutral points set with the pipette tool.

I did rotate the image so it was correctly oriented - i'm not sure what combination of rotation/flip it is though.

No iSRD, no USM, no AACO, no Multi-exposure, no Descreening.

I'm attaching a pdf of the scan settings created by silverfast with the "info" button.

Perhaps you could look in your LogFile as well for lines similar to these:
26.07.2012 00:21:53.796 sfDebug: Memory mapping limit: 512 MBytes
26.07.2012 00:21:53.796 sfDebug: Used RAM limit for mapping: 3072 MBytes
26.07.2012 00:23:57.693 sfDebug: CsfPreviewManager::processScanFrame : Message decided to use low mem mode as Image seems BIG

IMO, those three right there are the culprits responsible for all the behavior I'm experiencing.

Thank you for your help - I really appreciate it very much.