Gabriel Klyber
2016-01-14 04:36:56 UTC
I have been experimenting with Polipo on Windows using MinGW_w64. I
found that the following enables some native IPv6 functions in
winsock2 on Windows 8 and above:
#define WINVER 0x0603
I have made a number of source code tweaks, but found that yuuboshi's
patches covered most of what I modified.
I continued to have problems with unlinking files in the cache and
pages from slashdot.org failing to load (among other sites). It does
appear that some changes to unlinking code needs to be made. I started
to work on building a list of files ready to be unlinked and searching
the disk cache, but have not worked out all of the code for it.
Specifically, the rewriteEntry code in diskcache.c does some things
that will not work on Windows. Rather than move the entry to a
temporary file and rebuild it, I changed the code to just destroy the
file and create a new entry. So far, this has not executed, so it must
not be a very common case.
I played with settings and have found the following to be much more
stable than defaults:
#define CHUNK_SIZE 16384
chunkHighMark = 536870912
objectHighMark = 262144
maxDiskEntries = 1000
Windows has a file handle restriction of 2048 handles per process
using standard C I/O functions. It seems that with 1000
maxDiskEntries, the cache does not have unlinking errors. Will it
still write files out if it has the ability to keep so many open?
Increasing CHUNK_SIZE helped slashdot.org load and eliminated most
errors about failing to read from the server. I still occasionally get
errors about not being able to write to the client. CHUNK_SIZE of
32768 or 65536 seemed to work a little better for loading some pages,
but caused crashes in other cases (suspecting overflows in
calculations).
I tried setting higher chunkHighMark settings, but there were places
in the code that would overflow if it was set too high.
Either way, I have been very happy for the past few days with these
configuration changes. IPv6 has been stable, though I did have some
issues listening on an IPv6 address. My needs are met with just
listening on IPv4.
I am happy with the memory usage. If some of the variables were
64-bit, I suspect that the chunkHighMark and CHUNK_SIZE limitations
may be eliminated. The only concern I have is that file handles seem
to not be released. With such a high maxDiskEntries, does the cache
keep the files open until the max value? It would seem to make sense
to close them after some amount of time. Regardless, it is working
well enough that I may not mess with it for now.
Regards,
Gabe Klyber
found that the following enables some native IPv6 functions in
winsock2 on Windows 8 and above:
#define WINVER 0x0603
I have made a number of source code tweaks, but found that yuuboshi's
patches covered most of what I modified.
I continued to have problems with unlinking files in the cache and
pages from slashdot.org failing to load (among other sites). It does
appear that some changes to unlinking code needs to be made. I started
to work on building a list of files ready to be unlinked and searching
the disk cache, but have not worked out all of the code for it.
Specifically, the rewriteEntry code in diskcache.c does some things
that will not work on Windows. Rather than move the entry to a
temporary file and rebuild it, I changed the code to just destroy the
file and create a new entry. So far, this has not executed, so it must
not be a very common case.
I played with settings and have found the following to be much more
stable than defaults:
#define CHUNK_SIZE 16384
chunkHighMark = 536870912
objectHighMark = 262144
maxDiskEntries = 1000
Windows has a file handle restriction of 2048 handles per process
using standard C I/O functions. It seems that with 1000
maxDiskEntries, the cache does not have unlinking errors. Will it
still write files out if it has the ability to keep so many open?
Increasing CHUNK_SIZE helped slashdot.org load and eliminated most
errors about failing to read from the server. I still occasionally get
errors about not being able to write to the client. CHUNK_SIZE of
32768 or 65536 seemed to work a little better for loading some pages,
but caused crashes in other cases (suspecting overflows in
calculations).
I tried setting higher chunkHighMark settings, but there were places
in the code that would overflow if it was set too high.
Either way, I have been very happy for the past few days with these
configuration changes. IPv6 has been stable, though I did have some
issues listening on an IPv6 address. My needs are met with just
listening on IPv4.
I am happy with the memory usage. If some of the variables were
64-bit, I suspect that the chunkHighMark and CHUNK_SIZE limitations
may be eliminated. The only concern I have is that file handles seem
to not be released. With such a high maxDiskEntries, does the cache
keep the files open until the max value? It would seem to make sense
to close them after some amount of time. Regardless, it is working
well enough that I may not mess with it for now.
Regards,
Gabe Klyber